@devries-cameron20 / composio-1 / commits / adf6b02eb
Fork 0

Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

feat: show the model each tool's own JSON Schema in the Vercel, LlamaIndex, and Claude Agent SDK providers (#4940)

## Summary Zod-based providers rendered each tool's Zod schema back into JSON Schema for the model, which drops keywords Zod v3 cannot carry (`format: "binary"`, `contentEncoding`, `contentMediaType`, custom formats such as `repo.nwo`). The direct providers send the source schema, so the same tool looked different to the model depending on the provider. This moves the providers that can do it without changing what they accept: | Provider | Model sees | Arguments validated by | |---|---|---| | Vercel | the tool's JSON Schema (`$ref`s inlined) via the AI SDK's `jsonSchema(schema, { validate })`; in strict mode, the output of `toStrictJsonSchema` the same way | the same v3 Zod schema as before | | LlamaIndex | the tool's JSON Schema (`$ref`s inlined) as `FunctionTool`'s model-facing parameters | the v3 Zod schema, passed to the same constructor | | Claude Agent SDK | a schema built with `@composio/json-schema-to-zod/v4`, which forwards the keywords through `.meta()` | the same v4 schema | | LangChain | unchanged (v3) | unchanged | Core stays on the v3 builder: moving it to `/v4` would require zod `>=4.1.13` for every core user. LangChain and LlamaIndex render v4 schemas in output mode, which across the 930-tool set loses `default` (and makes the field required) in 397 tools, and LangChain also closes free-form objects. So LangChain stays on v3 until it can take a separate model-facing schema or render in input mode. Designated PR reviewer: [@jkomyno](mailto:alberto@composio.dev). Part of https://github.com/ComposioHQ/composio/issues/4911 ## Changes - **Vercel:** `jsonSchema()` with a `validate` backed by the existing Zod schema (available since `ai` 6.0.0, so the `^6 || ^7` range holds). The model now also sees `title`, `default`, `examples`, and OpenAPI `nullable`. In strict mode the strict JSON Schema is sent as is, so Zod no longer adds a `pattern` beside `email`, which OpenAI and Anthropic strict mode reject. `tool.inputSchema` is now an AI SDK `Schema` instead of a Zod schema (minor bump; the changeset shows `asSchema(tool.inputSchema).validate(value)` as the replacement for `.safeParse()`). - **LlamaIndex:** `new FunctionTool(execute, { parameters: jsonSchema }, zodValidator)`; 0.12.0 and 0.12.1 share `@llamaindex/core` 0.6.22. Duplicate `required` entries are now removed after `$ref`s are inlined, using `deduplicateJsonSchemaRequiredArrays` from core. That export first shipped in core 0.14.0, so LlamaIndex's verified core floor in the compatibility check moves from 0.11.0 to 0.14.0, the same floor Anthropic and Cloudflare use for it. The published `>=0.10.0` range is unchanged, like the other providers' floors. - **Claude Agent SDK:** builds with `/v4`; the `zod` peer moves from `^4.0.0` to `^4.1.13` (minor bump), and the package now depends on `@composio/json-schema-to-zod`. It needs the `/v4` release of the converter, whose changeset is already pending on `next`. - `ts/scripts/check-provider-compatibility.ts` packs `@composio/json-schema-to-zod` for providers that import it; without it the minimum lane fails on the unpublished `./v4` subpath. - Vercel AI SDK v6/v7 E2E fixtures check that the source schema reaches the AI SDK. ## Type of change - [ ] Bug fix - [x] New feature - [ ] Refactor/Chore - [ ] Documentation - [x] Breaking change ## How Has This Been Tested? - **Validation parity:** each provider passes all 53 cases and 134 instances of the shared corpus through its real validator (AI SDK `validate`, LlamaIndex's validator, MCP `tools/call` for Claude). For the Claude path, v3 and v4 agree on all 2,790 checks over the 930-tool set. - **New tests:** per provider, `format: "binary"`, `contentEncoding: "base16"`, and `contentMediaType` reach the model-facing schema; in Vercel strict mode `format: "uri"` is stripped and `email` is kept without a `pattern`. - **Live (12 requests):** gpt-5-mini through `@ai-sdk/openai` with the request body captured, and claude-haiku-4-5 with the schema the AI SDK holds, over a synthetic tool, `GITHUB_CREATE_A_PULL_REQUEST`, and `GITHUB_CREATE_A_CODESPACE_FOR_THE_AUTHENTICATED_USER`: - Non-strict: 200 for all six requests across both providers. The request bodies carried `binary`, `base16`, the media type, `repo.nwo`, and `title`. - Strict, OpenAI: 200, 200, 200. - Strict, Anthropic: 200, 200, and **400** on the codespace tool (see Additional context). - **Tests:** core 1884 passed (2 expected fail), json-schema-to-zod 2296, json-schema-to-effect-schema 156, vercel 71, llamaindex 78, claude-agent-sdk 79, langchain 15. - **Other checks:** - typecheck clean for every touched package; `pnpm lint:packages` exit 0; `pnpm validate:changesets` passed. - `pnpm test:provider-compatibility` 5 pass, and `pnpm check:provider-compatibility` passes every lane. - E2E: `vercel-ai-sdk-v6` 21 pass, `vercel-ai-sdk-v7` 24 pass, `claude-agent-sdk` 2 pass (live Anthropic). ## Checklist - [ ] I have read the Code of Conduct and this PR adheres to it - [ ] I have read and understand every line of this diff, including any LLM-generated code - [x] Any third-party docs links or promotional content follow the [third-party links policy](https://github.com/ComposioHQ/composio/blob/next/CONTRIBUTING.md#third-party-links-in-docs) - [x] I ran linters/tests locally and they passed - [x] I updated documentation as needed - [x] I added tests or explain why not applicable - [x] I added a changeset if this change affects published packages ## Additional context **Strict mode and Anthropic:** `VercelProvider({ strict: true })` rewrites each tool's schema into the strict form, but it does not set the AI SDK's per-tool `strict` flag, so neither OpenAI nor Anthropic is asked to enforce it. A live AI SDK 7 check with `GOOGLEDRIVE_CREATE_DRIVE` confirms this: without the flag, both vendors accept the request. With `strict: true` on the tool, Anthropic rejects it ("properties maximum, minimum are not supported") and OpenAI accepts it. So the Anthropic strict-mode rejections seen during development only appear when the flag is set on the tool by hand: - OpenAPI `nullable` on an object, now fixed by https://github.com/ComposioHQ/composio/pull/4997, which this branch includes. - Numeric `minimum`/`maximum`, which 23 of the 816 strict-capable tools carry. Setting the flag is not planned as part of this work. Anthropic strict mode also limits each request, across all strict tools, to 16 union-typed parameters and 24 optional ones, and rejects recursive definitions. The strict rewrite turns every optional parameter into a union, so a request with around ten Composio tools would exceed the limit.

Alberto Schiabel committed Oct 21, 2026 at 19:36 UTC adf6b02ebe82268c41fc250fd3c7eec2b981b54f
27 files changed +687 -100
.changeset/claude-agent-sdk-zod-v4-schemas.md new
+7
@@ -0,0 +1,7 @@
1 +---
2 +'@composio/claude-agent-sdk': minor
3 +---
4 +
5 +Build tool input schemas with the Zod v4 builder from `@composio/json-schema-to-zod/v4`, so `format`, `contentEncoding`, `contentMediaType`, and `type: "integer"` survive the SDK's conversion to the JSON Schema the model sees. Before, a field such as `{ type: "string", format: "binary" }` reached the model as `{ type: "string" }`. The values a tool accepts and rejects are unchanged.
6 +
7 +The `zod` peer dependency is now `^4.1.13` (was `^4.0.0`): the v4 builder needs the metadata registry zod shares across copies from 4.1.13 on, and throws at load time on an older zod. Upgrade zod to 4.1.13 or newer before upgrading this package.
.changeset/llamaindex-source-tool-schemas.md new
+5
@@ -0,0 +1,5 @@
1 +---
2 +'@composio/llamaindex': patch
3 +---
4 +
5 +Show the model each tool's JSON Schema as is. The provider now builds the `FunctionTool` with the JSON Schema as its parameters and the Zod schema as its validator, instead of letting LlamaIndex render the Zod schema back to JSON Schema. Keywords such as `format: "binary"`, `contentEncoding`, `contentMediaType`, and `title` reach the model. Internal `$ref`s are still inlined first, and the values a tool accepts and rejects are unchanged.
.changeset/vercel-source-tool-schemas.md new
+9
@@ -0,0 +1,9 @@
1 +---
2 +'@composio/vercel': minor
3 +---
4 +
5 +Show the model each tool's JSON Schema as is. The provider now passes the schema to the AI SDK with `jsonSchema()`, so keywords a Zod schema cannot carry, such as `format: "binary"`, `contentEncoding`, `contentMediaType`, `title`, `default`, and `examples`, reach the model. Internal `$ref`s are still inlined first. Tool-call arguments are validated with the same Zod schema as before, so the values a tool accepts and rejects are unchanged.
6 +
7 +In strict mode the model now sees the output of the strict rewrite itself rather than a Zod rendering of it. Zod adds a `pattern` next to formats such as `email`, and strict structured outputs reject the one it adds for `email`, so a strict request with an email parameter would have failed once Zod forwarded more of the schema.
8 +
9 +`tool.inputSchema` is now an AI SDK `Schema` instead of a Zod schema. Code that called `inputSchema.safeParse()` directly can validate with `asSchema(tool.inputSchema).validate(value)` from `ai`.
docs/content/docs/providers/vercel.mdx
+1 -1
@@ -68,7 +68,7 @@ const composio = new Composio({ provider: new VercelProvider({ strict: true }) }
68 ```
69
70 <Callout type="info">
71 -The provider converts each Composio tool's JSON Schema to a Zod schema for the AI SDK and normalizes tool arguments, so it still works when a model emits tool input as a JSON string rather than an object.
71 +The provider sends each Composio tool's JSON Schema to the model as is, so keywords such as `format`, `contentEncoding`, and `contentMediaType` reach it, and validates tool-call arguments with a Zod schema built from the same JSON Schema. It also normalizes tool arguments, so it still works when a model emits tool input as a JSON string rather than an object.
72 </Callout>
73
74 ## Next
pnpm-lock.yaml
+3
@@ -1547,6 +1547,9 @@ importers:
1547
1548 ts/packages/providers/claude-agent-sdk:
1549 dependencies:
1550 + '@composio/json-schema-to-zod':
1551 + specifier: workspace:*
1552 + version: link:../../json-schema-to-zod
1553 zod:
1554 specifier: ^4.5.4
1555 version: 4.5.4
test/provider-compatibility.test.ts
+1 -1
@@ -38,7 +38,7 @@ describe('provider compatibility plan', () => {
38 '@composio/cloudflare': '0.14.0',
39 '@composio/google': '0.16.0',
40 '@composio/langchain': '0.11.0',
41 - '@composio/llamaindex': '0.11.0',
41 + '@composio/llamaindex': '0.14.0',
42 '@composio/mastra': '0.18.0',
43 '@composio/openai': '0.18.0',
44 '@composio/openai-agents': '0.18.0',
ts/e2e-tests/runtimes/node/vercel-ai-sdk-v6/README.md
+8 -7
@@ -8,13 +8,14 @@ Verifies that `@composio/vercel` installs, typechecks, and executes wrapped tool
8
9 ## What It Tests
10
11 -| Marker | Description |
12 -| ----------------------------------------------- | ------------------------------------------------------------------- |
13 -| `vercel ai sdk compatibility typecheck passed` | The packed provider typechecks against `ai@6` (`fixtures/index.ts`) |
14 -| `WRAPPED_TOOL_INPUT_SCHEMA_OK` | `wrapTool` returns an AI SDK tool with `inputSchema` + `execute` |
15 -| `OBJECT_INPUT_EXECUTION_OK` | Object inputs are forwarded to `executeTool` |
16 -| `STRING_INPUT_EXECUTION_OK` | JSON-string inputs are normalized before `executeTool` |
17 -| `TOOL_SET_OK` | `wrapTools` produces a `ToolSet`-compatible collection |
11 +| Marker | Description |
12 +| ---------------------------------------------- | --------------------------------------------------------------------- |
13 +| `vercel ai sdk compatibility typecheck passed` | The packed provider typechecks against `ai@6` (`fixtures/index.ts`) |
14 +| `WRAPPED_TOOL_INPUT_SCHEMA_OK` | `wrapTool` returns an AI SDK tool with `inputSchema` + `execute` |
15 +| `OBJECT_INPUT_EXECUTION_OK` | Object inputs are forwarded to `executeTool` |
16 +| `STRING_INPUT_EXECUTION_OK` | JSON-string inputs are normalized before `executeTool` |
17 +| `SOURCE_SCHEMA_OK` | The source JSON Schema reaches `inputSchema`; Zod validates arguments |
18 +| `TOOL_SET_OK` | `wrapTools` produces a `ToolSet`-compatible collection |
19
20 ## Fixture
21
ts/e2e-tests/runtimes/node/vercel-ai-sdk-v6/e2e.test.ts
+4
@@ -47,6 +47,10 @@ e2e(import.meta.url, {
47 expect(result.stdout).toContain('STRING_INPUT_EXECUTION_OK');
48 });
49
50 + it('sends the source schema and validates with Zod', () => {
51 + expect(result.stdout).toContain('SOURCE_SCHEMA_OK');
52 + });
53 +
54 it('produces a ToolSet-compatible collection', () => {
55 expect(result.stdout).toContain('TOOL_SET_OK');
56 });
ts/e2e-tests/runtimes/node/vercel-ai-sdk-v6/fixtures/index.mjs
+35
@@ -1,4 +1,5 @@
1 import { VercelProvider } from '@composio/vercel';
2 +import { asSchema } from 'ai';
3
4 const composioTool = {
5 slug: 'TEST_TOOL',
@@ -49,6 +50,40 @@ if (calls.at(-1)?.slug !== 'TEST_TOOL' || calls.at(-1)?.params?.query !== 'strin
50 }
51 console.log('STRING_INPUT_EXECUTION_OK');
52
53 +// The model is shown the tool's JSON Schema as is, and the AI SDK validates
54 +// tool-call arguments with the Zod schema built from it.
55 +const annotatedTool = {
56 + ...composioTool,
57 + slug: 'ANNOTATED_TOOL',
58 + inputParameters: {
59 + type: 'object',
60 + properties: {
61 + file: { type: 'string', format: 'binary' },
62 + hex: {
63 + type: 'string',
64 + contentEncoding: 'base16',
65 + contentMediaType: 'application/octet-stream',
66 + },
67 + },
68 + required: ['hex'],
69 + },
70 +};
71 +const annotatedSchema = asSchema(provider.wrapTool(annotatedTool, executeTool).inputSchema);
72 +const annotatedJson = await annotatedSchema.jsonSchema;
73 +if (
74 + annotatedJson.properties?.file?.format !== 'binary' ||
75 + annotatedJson.properties?.hex?.contentEncoding !== 'base16' ||
76 + annotatedJson.properties?.hex?.contentMediaType !== 'application/octet-stream'
77 +) {
78 + throw new Error(`The source keywords did not reach the schema: ${JSON.stringify(annotatedJson)}`);
79 +}
80 +const accepted = await annotatedSchema.validate({ hex: 'ff' });
81 +const rejected = await annotatedSchema.validate({ hex: 1 });
82 +if (!accepted.success || rejected.success) {
83 + throw new Error('Tool-call arguments were not validated with the Zod schema');
84 +}
85 +console.log('SOURCE_SCHEMA_OK');
86 +
87 const toolSet = provider.wrapTools([composioTool], executeTool);
88 if (!toolSet.TEST_TOOL) {
89 throw new Error('Wrapped tools collection is missing TEST_TOOL');
ts/e2e-tests/runtimes/node/vercel-ai-sdk-v7/README.md
+9 -8
@@ -8,14 +8,15 @@ Verifies that `@composio/vercel` installs, typechecks, and executes wrapped tool
8
9 ## What It Tests
10
11 -| Marker | Description |
12 -| ----------------------------------------------- | ------------------------------------------------------------------- |
13 -| `vercel ai sdk compatibility typecheck passed` | The packed provider typechecks against `ai@7` (`fixtures/index.ts`) |
14 -| `WRAPPED_TOOL_INPUT_SCHEMA_OK` | `wrapTool` returns an AI SDK tool with `inputSchema` + `execute` |
15 -| `OBJECT_INPUT_EXECUTION_OK` | Object inputs are forwarded to `executeTool` |
16 -| `STRING_INPUT_EXECUTION_OK` | JSON-string inputs are normalized before `executeTool` |
17 -| `V7_EXECUTION_OPTIONS_OK` | A wrapped tool accepts AI SDK v7 `ToolExecutionOptions` |
18 -| `TOOL_SET_OK` | `wrapTools` produces a `ToolSet`-compatible collection |
11 +| Marker | Description |
12 +| ---------------------------------------------- | --------------------------------------------------------------------- |
13 +| `vercel ai sdk compatibility typecheck passed` | The packed provider typechecks against `ai@7` (`fixtures/index.ts`) |
14 +| `WRAPPED_TOOL_INPUT_SCHEMA_OK` | `wrapTool` returns an AI SDK tool with `inputSchema` + `execute` |
15 +| `OBJECT_INPUT_EXECUTION_OK` | Object inputs are forwarded to `executeTool` |
16 +| `STRING_INPUT_EXECUTION_OK` | JSON-string inputs are normalized before `executeTool` |
17 +| `V7_EXECUTION_OPTIONS_OK` | A wrapped tool accepts AI SDK v7 `ToolExecutionOptions` |
18 +| `SOURCE_SCHEMA_OK` | The source JSON Schema reaches `inputSchema`; Zod validates arguments |
19 +| `TOOL_SET_OK` | `wrapTools` produces a `ToolSet`-compatible collection |
20
21 ## Fixture
22
ts/e2e-tests/runtimes/node/vercel-ai-sdk-v7/e2e.test.ts
+4
@@ -51,6 +51,10 @@ e2e(import.meta.url, {
51 expect(result.stdout).toContain('V7_EXECUTION_OPTIONS_OK');
52 });
53
54 + it('sends the source schema and validates with Zod', () => {
55 + expect(result.stdout).toContain('SOURCE_SCHEMA_OK');
56 + });
57 +
58 it('produces a ToolSet-compatible collection', () => {
59 expect(result.stdout).toContain('TOOL_SET_OK');
60 });
ts/e2e-tests/runtimes/node/vercel-ai-sdk-v7/fixtures/index.mjs
+35
@@ -1,4 +1,5 @@
1 import { VercelProvider } from '@composio/vercel';
2 +import { asSchema } from 'ai';
3
4 const composioTool = {
5 slug: 'TEST_TOOL',
@@ -62,6 +63,40 @@ if (calls.at(-1)?.slug !== 'TEST_TOOL' || calls.at(-1)?.params?.query !== 'optio
63 }
64 console.log('V7_EXECUTION_OPTIONS_OK');
65
66 +// The model is shown the tool's JSON Schema as is, and the AI SDK validates
67 +// tool-call arguments with the Zod schema built from it.
68 +const annotatedTool = {
69 + ...composioTool,
70 + slug: 'ANNOTATED_TOOL',
71 + inputParameters: {
72 + type: 'object',
73 + properties: {
74 + file: { type: 'string', format: 'binary' },
75 + hex: {
76 + type: 'string',
77 + contentEncoding: 'base16',
78 + contentMediaType: 'application/octet-stream',
79 + },
80 + },
81 + required: ['hex'],
82 + },
83 +};
84 +const annotatedSchema = asSchema(provider.wrapTool(annotatedTool, executeTool).inputSchema);
85 +const annotatedJson = await annotatedSchema.jsonSchema;
86 +if (
87 + annotatedJson.properties?.file?.format !== 'binary' ||
88 + annotatedJson.properties?.hex?.contentEncoding !== 'base16' ||
89 + annotatedJson.properties?.hex?.contentMediaType !== 'application/octet-stream'
90 +) {
91 + throw new Error(`The source keywords did not reach the schema: ${JSON.stringify(annotatedJson)}`);
92 +}
93 +const accepted = await annotatedSchema.validate({ hex: 'ff' });
94 +const rejected = await annotatedSchema.validate({ hex: 1 });
95 +if (!accepted.success || rejected.success) {
96 + throw new Error('Tool-call arguments were not validated with the Zod schema');
97 +}
98 +console.log('SOURCE_SCHEMA_OK');
99 +
100 const toolSet = provider.wrapTools([composioTool], executeTool);
101 if (!toolSet.TEST_TOOL) {
102 throw new Error('Wrapped tools collection is missing TEST_TOOL');
ts/packages/providers/claude-agent-sdk/README.md
+2
@@ -55,6 +55,8 @@ For multi-turn conversations, store `session.sessionId` and reuse it with `compo
55
56 The Claude Agent SDK provides tools to agents through MCP (Model Context Protocol) servers. `ClaudeAgentSDKProvider` converts each Composio tool definition into an MCP tool and routes execution back through Composio, so `createSdkMcpServer` can serve them in process; no separate server to run.
57
58 +Each tool's input schema is built as a Zod v4 schema, which the SDK turns back into the JSON Schema the model sees. The provider carries `format`, `contentEncoding`, `contentMediaType`, and `type: "integer"` through that round trip, and validates tool-call arguments as the other Zod-based providers do. It requires zod 4.1.13 or newer.
59 +
60 ## Links
61
62 - [Composio quickstart](https://docs.composio.dev/docs/quickstart)
ts/packages/providers/claude-agent-sdk/package.json
+2 -1
@@ -50,12 +50,13 @@
50 ],
51 "license": "ISC",
52 "dependencies": {
53 + "@composio/json-schema-to-zod": "workspace:*",
54 "zod": "^4.5.4"
55 },
56 "peerDependencies": {
57 "@anthropic-ai/claude-agent-sdk": "^0.3.199",
58 "@composio/core": ">=0.10.0 <1.0.0 || >=1.0.0-beta.0 <1.0.0",
58 - "zod": "^4.0.0"
59 + "zod": "^4.1.13"
60 },
61 "devDependencies": {
62 "@anthropic-ai/claude-agent-sdk": "^0.3.295",
ts/packages/providers/claude-agent-sdk/src/index.ts
+23 -3
@@ -16,13 +16,33 @@ import {
16 McpServerGetResponse,
17 dereferenceJsonSchema,
18 normalizeToolArguments,
19 - jsonSchemaToZodSchema,
19 + JsonSchemaToZodError,
20 } from '@composio/core';
21 +import { jsonSchemaToZod, type JsonSchema } from '@composio/json-schema-to-zod/v4';
22 +import type { ZodType } from 'zod/v4';
23 import {
24 tool as sdkTool,
25 type Options as ClaudeAgentOptions,
26 } from '@anthropic-ai/claude-agent-sdk';
27
28 +/**
29 + * Builds the Zod v4 schema the SDK registers. The SDK renders it as the tool's
30 + * JSON Schema with `z.toJSONSchema`, and the v4 builder forwards `format`,
31 + * `contentEncoding`, `contentMediaType`, and `type: "integer"` into the
32 + * schema's metadata, so those keywords reach the model. It accepts and rejects
33 + * the same values as the v3 builder behind `jsonSchemaToZodSchema`.
34 + */
35 +const toInputSchema = (schema: Record<string, unknown>): ZodType => {
36 + try {
37 + return jsonSchemaToZod(schema as JsonSchema);
38 + } catch (error) {
39 + const detail = error instanceof Error ? `: ${error.message}` : '';
40 + throw new JsonSchemaToZodError(`Failed to convert JSON Schema to Zod Schema${detail}`, {
41 + cause: error,
42 + });
43 + }
44 +};
45 +
46 /**
47 * Type for a single Claude Agent SDK MCP tool definition
48 */
@@ -111,7 +131,7 @@ export class ClaudeAgentSDKProvider extends BaseAgenticProvider<
131 //
132 // A tool with no input parameters stays closed: the fallback spells out
133 // `additionalProperties: false`, because a bare `properties: {}` is now an open object.
114 - const inputZodSchema = jsonSchemaToZodSchema(
134 + const inputZodSchema = toInputSchema(
135 dereferenceJsonSchema(
136 composioTool.inputParameters ?? {
137 type: 'object',
@@ -128,7 +148,7 @@ export class ClaudeAgentSDKProvider extends BaseAgenticProvider<
148 const defineTool = sdkTool as unknown as (
149 name: string,
150 description: string,
131 - inputSchema: ReturnType<typeof jsonSchemaToZodSchema>,
151 + inputSchema: ZodType,
152 handler: (
153 args: Record<string, unknown>
154 ) => Promise<{ content: Array<{ type: 'text'; text: string }> }>
ts/packages/providers/claude-agent-sdk/test/claude-agent-sdk-schema.test.ts new
+125
@@ -0,0 +1,125 @@
1 +/**
2 + * The schema the provider shows the model through the Claude Agent SDK, and
3 + * the validation the SDK runs on tool-call arguments, both driven through the
4 + * real SDK over an in-memory MCP transport.
5 + */
6 +import { describe, expect, it, vi } from 'vitest';
7 +import { Client } from '@modelcontextprotocol/sdk/client/index.js';
8 +import { InMemoryTransport } from '@modelcontextprotocol/sdk/inMemory.js';
9 +import { createSdkMcpServer } from '@anthropic-ai/claude-agent-sdk';
10 +import type { Tool, GlobalExecuteToolFn } from '@composio/core';
11 +import { ClaudeAgentSDKProvider } from '../src';
12 +import {
13 + acceptedFor,
14 + loadObjectCases,
15 +} from '../../../core/test/fixtures/json-schema-conversion/corpus';
16 +
17 +type Json = Record<string, unknown>;
18 +
19 +const buildTool = (inputParameters: Json): Tool =>
20 + ({
21 + slug: 'TEST_TOOL',
22 + name: 'Test Tool',
23 + description: 'A tool under test',
24 + version: '20260901_00',
25 + availableVersions: ['20260901_00'],
26 + tags: [],
27 + inputParameters,
28 + }) as unknown as Tool;
29 +
30 +const register = async (composioTool: Tool) => {
31 + const executeToolFn = vi.fn().mockResolvedValue({
32 + data: {},
33 + error: null,
34 + successful: true,
35 + }) as unknown as GlobalExecuteToolFn;
36 + const provider = new ClaudeAgentSDKProvider();
37 + const server = createSdkMcpServer({
38 + name: 'composio-test',
39 + version: '1.0.0',
40 + tools: [provider.wrapTool(composioTool, executeToolFn)],
41 + });
42 + const [clientTransport, serverTransport] = InMemoryTransport.createLinkedPair();
43 + const client = new Client({ name: 'test-client', version: '1.0.0' });
44 + await server.instance.server.connect(serverTransport);
45 + await client.connect(clientTransport);
46 + const { tools } = await client.listTools();
47 + return {
48 + client,
49 + inputSchema: tools[0].inputSchema as Json,
50 + accepts: async (args: unknown) => {
51 + const result = (await client.callTool({
52 + name: composioTool.slug,
53 + arguments: args as Record<string, unknown>,
54 + })) as { isError?: boolean };
55 + return !(result.isError ?? false);
56 + },
57 + };
58 +};
59 +
60 +describe('ClaudeAgentSDKProvider schema sent to the model', () => {
61 + it('sends the source keywords a v3 Zod schema could not carry', async () => {
62 + const { client, inputSchema } = await register(
63 + buildTool({
64 + type: 'object',
65 + properties: {
66 + file: { type: 'string', format: 'binary' },
67 + hex: {
68 + type: 'string',
69 + contentEncoding: 'base16',
70 + contentMediaType: 'application/octet-stream',
71 + },
72 + count: { type: 'integer', minimum: 0 },
73 + },
74 + required: ['file'],
75 + })
76 + );
77 + await client.close();
78 + const properties = inputSchema.properties as Record<string, Json>;
79 +
80 + expect(properties.file).toEqual({ type: 'string', format: 'binary' });
81 + expect(properties.hex).toEqual({
82 + type: 'string',
83 + contentEncoding: 'base16',
84 + contentMediaType: 'application/octet-stream',
85 + });
86 + expect(properties.count).toMatchObject({ type: 'integer', minimum: 0 });
87 + });
88 +
89 + it('keeps a defaulted field optional and shows its default', async () => {
90 + const { client, inputSchema, accepts } = await register(
91 + buildTool({
92 + type: 'object',
93 + properties: {
94 + query: { type: 'string' },
95 + page: { type: 'integer', default: 1 },
96 + },
97 + required: ['query'],
98 + })
99 + );
100 + const acceptsWithoutPage = await accepts({ query: 'q' });
101 + await client.close();
102 +
103 + expect(inputSchema.required).toEqual(['query']);
104 + expect((inputSchema.properties as Record<string, Json>).page).toMatchObject({
105 + type: 'integer',
106 + default: 1,
107 + });
108 + expect(acceptsWithoutPage).toBe(true);
109 + });
110 +});
111 +
112 +describe('ClaudeAgentSDKProvider tool-call validation', () => {
113 + for (const testCase of loadObjectCases()) {
114 + it(`matches the shared corpus on ${testCase.id}`, async () => {
115 + const { client, accepts } = await register(buildTool(testCase.schema));
116 +
117 + for (const instance of testCase.instances) {
118 + expect(await accepts(instance.input), JSON.stringify(instance.input)).toBe(
119 + acceptedFor(instance, 'zod')
120 + );
121 + }
122 + await client.close();
123 + });
124 + }
125 +});
ts/packages/providers/llamaindex/README.md
+1 -1
@@ -40,7 +40,7 @@ const result = await myAgent.run(
40 console.log(result.data.result);
41 ```
42
43 -The provider converts each tool's JSON Schema parameters to Zod and wires execution through Composio, so LlamaIndex can call the tools directly.
43 +The provider gives LlamaIndex each tool's JSON Schema as the parameters the model sees, with internal `$ref`s inlined, and validates tool-call arguments with a Zod schema built from the same JSON Schema. Execution is wired through Composio, so LlamaIndex can call the tools directly.
44
45 ## Links
46
ts/packages/providers/llamaindex/src/index.ts
+26 -13
@@ -12,13 +12,17 @@ import {
12 ExecuteToolFn,
13 McpServerGetResponse,
14 McpUrlResponse,
15 + deduplicateJsonSchemaRequiredArrays,
16 dereferenceJsonSchema,
17 jsonSchemaToZodSchema,
18 normalizeToolArguments,
19 } from '@composio/core';
19 -import { tool as createLlamaindexTool, JSONValue } from 'llamaindex';
20 +import { FunctionTool, type tool as createLlamaindexTool } from 'llamaindex';
21
22 export type LlamaindexTool = ReturnType<typeof createLlamaindexTool>;
23 +type LlamaindexToolParameters = ConstructorParameters<
24 + typeof FunctionTool<Record<string, unknown>, Promise<string>>
25 +>[1]['parameters'];
26 export type LlamaindexToolCollection = Array<LlamaindexTool>;
27
28 export class LlamaindexProvider extends BaseAgenticProvider<
@@ -34,20 +38,29 @@ export class LlamaindexProvider extends BaseAgenticProvider<
38 * @returns The wrapped tool.
39 */
40 wrapTool(tool: Tool, executeTool: ExecuteToolFn): LlamaindexTool {
37 - const inputParams = tool.inputParameters;
38 - const inputParametersSchema = jsonSchemaToZodSchema(
39 - dereferenceJsonSchema(inputParams ?? {}, { onUnresolved: 'sentinel' })
41 + // The model sees the tool's JSON Schema as is, so keywords a Zod schema
42 + // cannot carry (`format`, `contentEncoding`, `contentMediaType`, `default`,
43 + // ...) reach it; LlamaIndex validates the arguments with the Zod schema
44 + // built from the same JSON Schema. The Zod converter does not follow $ref,
45 + // so inline the definitions first, then canonicalize the required arrays
46 + // the inlined definitions brought along.
47 + const parameters = deduplicateJsonSchemaRequiredArrays(
48 + dereferenceJsonSchema(tool.inputParameters ?? {}, { onUnresolved: 'sentinel' })
49 );
41 - return createLlamaindexTool({
42 - name: tool.slug,
43 - description: tool.description ?? tool.name ?? '',
44 - parameters: inputParametersSchema,
45 - execute: async input => {
46 - // Models occasionally emit tool input as a JSON string rather than an object (issue #2406).
47 - const result = await executeTool(tool.slug, normalizeToolArguments(input, tool.slug));
48 - return JSON.stringify(result);
50 + const execute = async (input: Record<string, unknown>) => {
51 + // Models occasionally emit tool input as a JSON string rather than an object (issue #2406).
52 + const result = await executeTool(tool.slug, normalizeToolArguments(input, tool.slug));
53 + return JSON.stringify(result);
54 + };
55 + return new FunctionTool(
56 + execute,
57 + {
58 + name: tool.slug,
59 + description: tool.description ?? tool.name ?? '',
60 + parameters: parameters as LlamaindexToolParameters,
61 },
50 - });
62 + jsonSchemaToZodSchema(parameters)
63 + ) as unknown as LlamaindexTool;
64 }
65
66 /**
ts/packages/providers/llamaindex/test/llamaindex-ref.test.ts
+26 -13
@@ -12,17 +12,13 @@ import { LlamaindexProvider } from '../src';
12 // Mock the llamaindex module (per-file scoping keeps this independent of llamaindex.test.ts).
13 vi.mock('llamaindex', () => {
14 return {
15 - tool: vi.fn().mockImplementation(config => ({
16 - metadata: {
17 - name: config.name,
18 - description: config.description,
19 - parameters: config.parameters,
20 - },
21 - call: config.execute,
22 - _type: 'llamaindex_tool',
23 - _config: config,
24 - })),
25 - JSONValue: {},
15 + FunctionTool: class {
16 + constructor(
17 + readonly call: (input: unknown) => unknown,
18 + readonly metadata: { name: string; description: string; parameters: unknown },
19 + readonly zodType: unknown
20 + ) {}
21 + },
22 };
23 });
24
@@ -97,9 +93,9 @@ describe('LlamaindexProvider regression: $ref in JSON Schema', () => {
93 executeToolFn = vi.fn().mockResolvedValue({ data: {}, error: null, successful: true });
94 });
95
100 - it('produces a Zod schema that validates the $defs-described shape', () => {
96 + it('validates arguments against the $defs-described shape', () => {
97 const wrapped = provider.wrapTool(refTool, executeToolFn);
102 - const schema = wrapped.metadata.parameters as {
98 + const schema = (wrapped as unknown as { zodType: unknown }).zodType as {
99 safeParse: (value: unknown) => { success: boolean };
100 };
101
@@ -107,6 +103,23 @@ describe('LlamaindexProvider regression: $ref in JSON Schema', () => {
103 expect(schema.safeParse({ message: 42 }).success).toBe(false);
104 });
105
106 + it('shows the model the schema with its definitions inlined', () => {
107 + const wrapped = provider.wrapTool(refTool, executeToolFn);
108 +
109 + expect(wrapped.metadata.parameters).toEqual({
110 + type: 'object',
111 + properties: {
112 + message: {
113 + type: 'object',
114 + properties: { subject: { type: 'string' }, body: { type: 'string' } },
115 + required: ['subject', 'body'],
116 + additionalProperties: false,
117 + },
118 + },
119 + required: ['message'],
120 + });
121 + });
122 +
123 it('does not throw when $ref points into an undeclared $defs block', () => {
124 expect(() => provider.wrapTool(danglingRefTool, executeToolFn)).not.toThrow();
125 });
ts/packages/providers/llamaindex/test/llamaindex-schema.test.ts new
+118
@@ -0,0 +1,118 @@
1 +/**
2 + * The schema the provider shows the model through LlamaIndex, and the
3 + * validation LlamaIndex runs on tool-call arguments.
4 + */
5 +import { describe, expect, it, vi } from 'vitest';
6 +import type { Tool } from '@composio/core';
7 +import { LlamaindexProvider } from '../src';
8 +import {
9 + acceptedFor,
10 + loadObjectCases,
11 +} from '../../../core/test/fixtures/json-schema-conversion/corpus';
12 +
13 +type Json = Record<string, unknown>;
14 +type Validator = { safeParse: (value: unknown) => { success: boolean } };
15 +
16 +// The real `FunctionTool`, which also keeps the validator it was given so the
17 +// tests can run it: LlamaIndex itself only logs a failed validation.
18 +vi.mock('llamaindex', async importOriginal => {
19 + const actual = await importOriginal<typeof import('llamaindex')>();
20 + class InspectableFunctionTool extends actual.FunctionTool<unknown, string> {
21 + readonly validator: unknown;
22 + constructor(...args: ConstructorParameters<typeof actual.FunctionTool<unknown, string>>) {
23 + super(...args);
24 + this.validator = args[2];
25 + }
26 + }
27 + return { ...actual, FunctionTool: InspectableFunctionTool };
28 +});
29 +
30 +const buildTool = (inputParameters: Json): Tool =>
31 + ({
32 + slug: 'TEST_TOOL',
33 + name: 'Test Tool',
34 + description: 'A tool under test',
35 + version: '20260901_00',
36 + availableVersions: ['20260901_00'],
37 + tags: [],
38 + inputParameters,
39 + }) as unknown as Tool;
40 +
41 +const wrap = (tool: Tool) =>
42 + new LlamaindexProvider().wrapTool(tool, async () => ({
43 + data: {},
44 + error: null,
45 + successful: true,
46 + }));
47 +
48 +describe('LlamaindexProvider schema sent to the model', () => {
49 + it('sends the source schema, including keywords a Zod schema cannot carry', () => {
50 + const inputParameters = {
51 + type: 'object',
52 + properties: {
53 + file: { type: 'string', format: 'binary' },
54 + hex: {
55 + type: 'string',
56 + contentEncoding: 'base16',
57 + contentMediaType: 'application/octet-stream',
58 + },
59 + page: { type: 'integer', default: 1 },
60 + },
61 + required: ['file'],
62 + };
63 +
64 + expect(wrap(buildTool(inputParameters)).metadata.parameters).toEqual(inputParameters);
65 + });
66 +
67 + it('removes duplicate required entries, including in inlined definitions', () => {
68 + const parameters = wrap(
69 + buildTool({
70 + type: 'object',
71 + properties: { query: { type: 'string' }, filter: { $ref: '#/$defs/Filter' } },
72 + required: ['query', 'query'],
73 + $defs: {
74 + Filter: {
75 + type: 'object',
76 + properties: { field: { type: 'string' } },
77 + required: ['field', 'field'],
78 + },
79 + },
80 + })
81 + ).metadata.parameters as Json;
82 +
83 + expect(parameters.required).toEqual(['query']);
84 + expect((parameters.properties as Record<string, Json>).filter.required).toEqual(['field']);
85 + });
86 +});
87 +
88 +describe('LlamaindexProvider tool-call validation', () => {
89 + for (const testCase of loadObjectCases()) {
90 + it(`matches the shared corpus on ${testCase.id}`, () => {
91 + const { validator } = wrap(buildTool(testCase.schema)) as unknown as {
92 + validator: Validator;
93 + };
94 +
95 + for (const instance of testCase.instances) {
96 + expect(validator.safeParse(instance.input).success, JSON.stringify(instance.input)).toBe(
97 + acceptedFor(instance, 'zod')
98 + );
99 + }
100 + });
101 + }
102 +
103 + it('passes the parsed arguments to the tool', async () => {
104 + const executeTool = vi.fn().mockResolvedValue({ data: {}, error: null, successful: true });
105 + const wrapped = new LlamaindexProvider().wrapTool(
106 + buildTool({
107 + type: 'object',
108 + properties: { query: { type: 'string' }, page: { type: 'integer', default: 1 } },
109 + required: ['query'],
110 + }),
111 + executeTool
112 + );
113 +
114 + await wrapped.call({ query: 'composio' });
115 +
116 + expect(executeTool).toHaveBeenCalledWith('TEST_TOOL', { query: 'composio', page: 1 });
117 + });
118 +});
ts/packages/providers/llamaindex/test/llamaindex.test.ts
+7 -12
@@ -5,18 +5,13 @@ import { Tool } from '@composio/core';
5 // Mock the llamaindex module
6 vi.mock('llamaindex', () => {
7 return {
8 - tool: vi.fn().mockImplementation(config => ({
9 - metadata: {
10 - name: config.name,
11 - description: config.description,
12 - parameters: config.parameters,
13 - },
14 - call: config.execute,
15 - // Mock LlamaIndex tool properties
16 - _type: 'llamaindex_tool',
17 - _config: config,
18 - })),
19 - JSONValue: {},
8 + FunctionTool: class {
9 + constructor(
10 + readonly call: (input: unknown) => unknown,
11 + readonly metadata: { name: string; description: string; parameters: unknown },
12 + readonly zodType: unknown
13 + ) {}
14 + },
15 };
16 });
17
ts/packages/providers/vercel/README.md
+4
@@ -51,6 +51,10 @@ Some models reject tool schemas that contain optional parameters. Pass `strict:
51 const composio = new Composio({ provider: new VercelProvider({ strict: true }) });
52 ```
53
54 +## Tool schemas
55 +
56 +The provider passes each tool's JSON Schema to the AI SDK with `jsonSchema()`, so the model sees it as is, including keywords such as `format`, `contentEncoding`, and `contentMediaType`. Internal `$ref`s are inlined first. Tool-call arguments are validated with a Zod schema built from the same JSON Schema. In strict mode the model sees the strict rewrite instead, which drops string formats strict structured outputs reject.
57 +
58 ## Links
59
60 - [Vercel AI SDK provider docs](https://docs.composio.dev/docs/providers/vercel)
ts/packages/providers/vercel/src/index.ts
+25 -5
@@ -25,7 +25,25 @@ import {
25 omitNullToolArguments,
26 } from '@composio/core';
27 import type { ToolSet as VercelToolSet, Tool as VercelTool } from 'ai';
28 -import { tool } from 'ai';
28 +import { jsonSchema, tool } from 'ai';
29 +
30 +type AiJsonSchema = Parameters<typeof jsonSchema>[0];
31 +
32 +/**
33 + * The AI SDK schema for a tool input: the model is shown `schema` itself, and
34 + * tool-call arguments are validated with the Zod schema built from it.
35 + */
36 +const toolInputSchema = (schema: Record<string, unknown>) => {
37 + const validator = jsonSchemaToZodSchema(schema);
38 + return jsonSchema(schema as AiJsonSchema, {
39 + validate: async value => {
40 + const result = await validator.safeParseAsync(value);
41 + return result.success
42 + ? { success: true, value: result.data }
43 + : { success: false, error: result.error };
44 + },
45 + });
46 +};
47
48 export type VercelToolCollection = VercelToolSet;
49 export class VercelProvider extends BaseAgenticProvider<
@@ -147,15 +165,17 @@ export class VercelProvider extends BaseAgenticProvider<
165 }
166 }
167
150 - // The Zod converter does not follow $ref, so inline the definitions the
151 - // strict rewrite keeps (cycles fall back to the permissive sentinel).
152 - const inputParametersSchema = jsonSchemaToZodSchema(
168 + // The model sees the tool's JSON Schema as is, so keywords a Zod schema
169 + // cannot carry (`format`, `contentEncoding`, `contentMediaType`, `title`,
170 + // ...) reach it. The Zod converter does not follow $ref, so inline the
171 + // definitions first (cycles fall back to the permissive sentinel).
172 + const inputSchema = toolInputSchema(
173 dereferenceJsonSchema(parameters, { onUnresolved: 'sentinel' })
174 );
175
176 return tool({
177 description: composioTool.description,
158 - inputSchema: inputParametersSchema,
178 + inputSchema,
179
180 execute: async params => {
181 // Models occasionally emit tool input as a JSON string rather than an object (issue #2406).
ts/packages/providers/vercel/test/vercel-ref.test.ts
+14 -10
@@ -16,8 +16,12 @@ interface MockedVercelTool {
16 _isMockedVercelTool: boolean;
17 }
18
19 +/** The AI SDK schema the provider builds; `validate` runs the tool's Zod schema. */
20 +type ValidatingSchema = { validate: (value: unknown) => Promise<{ success: boolean }> };
21 +
22 // Mock the ai module (per-file scoping keeps this independent of vercel.test.ts).
20 -vi.mock('ai', () => {
23 +vi.mock('ai', async importOriginal => {
24 + const actual = await importOriginal<typeof import('ai')>();
25 return {
26 tool: vi.fn().mockImplementation(toolConfig => {
27 return {
@@ -25,7 +29,7 @@ vi.mock('ai', () => {
29 _isMockedVercelTool: true,
30 } as MockedVercelTool;
31 }),
28 - jsonSchema: vi.fn().mockImplementation(schema => schema),
32 + jsonSchema: actual.jsonSchema,
33 };
34 });
35
@@ -101,21 +105,21 @@ describe('VercelProvider regression: $ref in JSON Schema', () => {
105 vi.clearAllMocks();
106 });
107
104 - it('produces a Zod schema that validates the $defs-described shape (default path)', () => {
108 + it('validates the $defs-described shape (default path)', async () => {
109 const wrapped = provider.wrapTool(refTool, executeToolFn) as unknown as MockedVercelTool;
106 - const schema = wrapped.inputSchema as { safeParse: (value: unknown) => { success: boolean } };
110 + const schema = wrapped.inputSchema as ValidatingSchema;
111
108 - expect(schema.safeParse({ message: { subject: 's', body: 'b' } }).success).toBe(true);
109 - expect(schema.safeParse({ message: 42 }).success).toBe(false);
112 + expect((await schema.validate({ message: { subject: 's', body: 'b' } })).success).toBe(true);
113 + expect((await schema.validate({ message: 42 })).success).toBe(false);
114 });
115
112 - it('produces a Zod schema that validates the $defs-described shape (strict path)', () => {
116 + it('validates the $defs-described shape (strict path)', async () => {
117 const strictProvider = new VercelProvider({ strict: true });
118 const wrapped = strictProvider.wrapTool(refTool, executeToolFn) as unknown as MockedVercelTool;
115 - const schema = wrapped.inputSchema as { safeParse: (value: unknown) => { success: boolean } };
119 + const schema = wrapped.inputSchema as ValidatingSchema;
120
117 - expect(schema.safeParse({ message: { subject: 's', body: 'b' } }).success).toBe(true);
118 - expect(schema.safeParse({ message: 42 }).success).toBe(false);
121 + expect((await schema.validate({ message: { subject: 's', body: 'b' } })).success).toBe(true);
122 + expect((await schema.validate({ message: 42 })).success).toBe(false);
123 });
124
125 it('does not throw when $ref points into an undeclared $defs block', () => {
ts/packages/providers/vercel/test/vercel-schema.test.ts new
+138
@@ -0,0 +1,138 @@
1 +/**
2 + * The schema the provider shows the model, rendered by the real AI SDK, and
3 + * the validation the AI SDK runs on tool-call arguments.
4 + *
5 + * The sibling suites mock `tool()`, so they see what the provider passes but
6 + * not the JSON Schema the AI SDK sends. These tests check that schema itself:
7 + * the source keywords reach the model, and strict mode sends only what strict
8 + * structured outputs accept.
9 + */
10 +import { asSchema } from 'ai';
11 +import { describe, expect, it } from 'vitest';
12 +import type { Tool } from '@composio/core';
13 +import { VercelProvider } from '../src';
14 +import {
15 + acceptedFor,
16 + loadObjectCases,
17 +} from '../../../core/test/fixtures/json-schema-conversion/corpus';
18 +
19 +type Json = Record<string, unknown>;
20 +
21 +const STRICT_STRING_FORMATS = [
22 + 'date-time',
23 + 'time',
24 + 'date',
25 + 'duration',
26 + 'email',
27 + 'hostname',
28 + 'ipv4',
29 + 'ipv6',
30 + 'uuid',
31 +];
32 +
33 +const buildTool = (inputParameters: Json): Tool =>
34 + ({
35 + slug: 'TEST_TOOL',
36 + name: 'Test Tool',
37 + description: 'A tool under test',
38 + version: '20260901_00',
39 + availableVersions: ['20260901_00'],
40 + tags: [],
41 + inputParameters,
42 + }) as unknown as Tool;
43 +
44 +const annotatedTool = buildTool({
45 + type: 'object',
46 + properties: {
47 + file: { type: 'string', format: 'binary', title: 'File' },
48 + hex: {
49 + type: 'string',
50 + contentEncoding: 'base16',
51 + contentMediaType: 'application/octet-stream',
52 + },
53 + link: { type: 'string', format: 'uri' },
54 + mail: { type: 'string', format: 'email' },
55 + when: { type: 'string', format: 'date-time' },
56 + links: { type: 'array', items: { type: 'string', format: 'uri' } },
57 + },
58 + required: ['file', 'mail'],
59 +});
60 +
61 +const wrap = (tool: Tool, strict: boolean) =>
62 + asSchema(
63 + new VercelProvider({ strict }).wrapTool(tool, async () => ({
64 + data: {},
65 + error: null,
66 + successful: true,
67 + })).inputSchema
68 + );
69 +
70 +const render = async (tool: Tool, strict: boolean): Promise<Json> =>
71 + (await wrap(tool, strict).jsonSchema) as Json;
72 +
73 +/** Every schema node of `schema`, including nested properties, items, and branches. */
74 +const nodes = (schema: unknown): Json[] => {
75 + if (typeof schema !== 'object' || schema === null) return [];
76 + if (Array.isArray(schema)) return schema.flatMap(nodes);
77 + const node = schema as Json;
78 + return [
79 + node,
80 + ...Object.values((node.properties as Json | undefined) ?? {}).flatMap(nodes),
81 + ...nodes(node.items),
82 + ...nodes(node.anyOf),
83 + ...nodes(node.oneOf),
84 + ...nodes(node.allOf),
85 + ];
86 +};
87 +
88 +describe('VercelProvider schema sent to the model', () => {
89 + it('sends the source keywords a Zod schema cannot carry', async () => {
90 + const properties = (await render(annotatedTool, false)).properties as Record<string, Json>;
91 +
92 + expect(properties.file).toEqual({ type: 'string', format: 'binary', title: 'File' });
93 + expect(properties.hex).toEqual({
94 + type: 'string',
95 + contentEncoding: 'base16',
96 + contentMediaType: 'application/octet-stream',
97 + });
98 + expect(properties.mail).toEqual({ type: 'string', format: 'email' });
99 + });
100 +
101 + it('carries only the string formats strict structured outputs accept under strict mode', async () => {
102 + const schema = await render(annotatedTool, true);
103 + const properties = schema.properties as Record<string, Json>;
104 + const formats = nodes(schema)
105 + .map(node => node.format)
106 + .filter(format => format !== undefined);
107 +
108 + expect(properties.link).not.toHaveProperty('format');
109 + expect(properties.file).not.toHaveProperty('format');
110 + expect(properties.mail).toMatchObject({ format: 'email' });
111 + expect(properties.hex).toMatchObject({
112 + contentEncoding: 'base16',
113 + contentMediaType: 'application/octet-stream',
114 + });
115 + for (const format of formats) {
116 + expect(STRICT_STRING_FORMATS).toContain(format);
117 + }
118 + });
119 +
120 + it('adds no pattern the source schema does not declare under strict mode', async () => {
121 + const patterns = nodes(await render(annotatedTool, true)).filter(node => 'pattern' in node);
122 +
123 + expect(patterns).toEqual([]);
124 + });
125 +});
126 +
127 +describe('VercelProvider tool-call validation', () => {
128 + for (const testCase of loadObjectCases()) {
129 + it(`matches the shared corpus on ${testCase.id}`, async () => {
130 + const schema = wrap(buildTool(testCase.schema), false);
131 +
132 + for (const instance of testCase.instances) {
133 + const result = await schema.validate!(instance.input);
134 + expect(result.success, JSON.stringify(instance.input)).toBe(acceptedFor(instance, 'zod'));
135 + }
136 + });
137 + }
138 +});
ts/packages/providers/vercel/test/vercel.test.ts
+32 -23
@@ -11,8 +11,12 @@ interface MockedVercelTool {
11 _isMockedVercelTool: boolean;
12 }
13
14 +/** The AI SDK schema the provider builds; `validate` runs the tool's Zod schema. */
15 +type ValidatingSchema = { validate: (value: unknown) => Promise<{ success: boolean }> };
16 +
17 // Mock the ai module
15 -vi.mock('ai', () => {
18 +vi.mock('ai', async importOriginal => {
19 + const actual = await importOriginal<typeof import('ai')>();
20 return {
21 tool: vi.fn().mockImplementation(toolConfig => {
22 return {
@@ -20,7 +24,7 @@ vi.mock('ai', () => {
24 _isMockedVercelTool: true,
25 } as MockedVercelTool;
26 }),
23 - jsonSchema: vi.fn().mockImplementation(schema => schema),
27 + jsonSchema: actual.jsonSchema,
28 };
29 });
30
@@ -82,11 +86,10 @@ describe('VercelProvider', () => {
86
87 expect(tool).toHaveBeenCalledWith({
88 description: mockTool.description,
85 - inputSchema: expect.any(Object), // Now it's a Zod schema object
89 + inputSchema: expect.any(Object),
90 execute: expect.any(Function),
91 });
92
89 - // jsonSchema is no longer called since we use Zod schemas directly
93 expect(wrapped._isMockedVercelTool).toBe(true);
94 });
95
@@ -101,10 +104,9 @@ describe('VercelProvider', () => {
104 mockExecuteToolFn
105 ) as unknown as MockedVercelTool;
106
104 - // Verify that tool was called with an empty object converted to Zod schema
107 expect(tool).toHaveBeenCalledWith({
108 description: toolWithoutParams.description,
107 - inputSchema: expect.any(Object), // Should be a Zod schema for empty object
109 + inputSchema: expect.any(Object),
110 execute: expect.any(Function),
111 });
112 expect(wrapped._isMockedVercelTool).toBe(true);
@@ -134,7 +136,7 @@ describe('VercelProvider', () => {
136 expect(mockExecuteToolFn).toHaveBeenCalledWith(mockTool.slug, params);
137 });
138
137 - it('keeps optional and nested parameters available under strict mode', () => {
139 + it('keeps optional and nested parameters available under strict mode', async () => {
140 const strictProvider = new VercelProvider({ strict: true });
141 const wrapped = strictProvider.wrapTool(
142 {
@@ -156,24 +158,31 @@ describe('VercelProvider', () => {
158 mockExecuteToolFn
159 ) as unknown as MockedVercelTool;
160
159 - const schema = wrapped.inputSchema as { safeParse: (value: unknown) => { success: boolean } };
161 + const schema = wrapped.inputSchema as ValidatingSchema;
162
163 // Optional parameters are still accepted, and may be null.
164 expect(
163 - schema.safeParse({ cfg: { url: 'https://example.com', note: 'x' }, id: null, label: null })
164 - .success
165 + (
166 + await schema.validate({
167 + cfg: { url: 'https://example.com', note: 'x' },
168 + id: null,
169 + label: null,
170 + })
171 + ).success
172 ).toBe(true);
173 // Every property is required under strict mode, so omitting one fails.
167 - expect(schema.safeParse({ cfg: { url: 'https://example.com' }, id: 'a' }).success).toBe(
168 - false
169 - );
174 + expect(
175 + (await schema.validate({ cfg: { url: 'https://example.com' }, id: 'a' })).success
176 + ).toBe(false);
177 // Objects are closed, so unknown keys fail.
178 expect(
172 - schema.safeParse({
173 - cfg: { url: 'https://example.com', note: null, extra: 1 },
174 - id: 'a',
175 - label: null,
176 - }).success
179 + (
180 + await schema.validate({
181 + cfg: { url: 'https://example.com', note: null, extra: 1 },
182 + id: 'a',
183 + label: null,
184 + })
185 + ).success
186 ).toBe(false);
187 });
188
@@ -208,7 +217,7 @@ describe('VercelProvider', () => {
217 });
218 });
219
211 - it('keeps the original schema for tools strict mode cannot express', () => {
220 + it('keeps the original schema for tools strict mode cannot express', async () => {
221 const strictProvider = new VercelProvider({ strict: true });
222 const wrapped = strictProvider.wrapTool(
223 {
@@ -225,10 +234,10 @@ describe('VercelProvider', () => {
234 mockExecuteToolFn
235 ) as unknown as MockedVercelTool;
236
228 - const schema = wrapped.inputSchema as { safeParse: (value: unknown) => { success: boolean } };
237 + const schema = wrapped.inputSchema as ValidatingSchema;
238
239 // `name` stays optional and `headers` still accepts arbitrary keys.
231 - expect(schema.safeParse({ headers: { 'x-a': '1' } }).success).toBe(true);
240 + expect((await schema.validate({ headers: { 'x-a': '1' } })).success).toBe(true);
241 });
242 });
243
@@ -252,12 +261,12 @@ describe('VercelProvider', () => {
261 expect(tool).toHaveBeenCalledTimes(2);
262 expect(tool).toHaveBeenCalledWith({
263 description: mockTool.description,
255 - inputSchema: expect.any(Object), // Now it's a Zod schema object
264 + inputSchema: expect.any(Object),
265 execute: expect.any(Function),
266 });
267 expect(tool).toHaveBeenCalledWith({
268 description: anotherTool.description,
260 - inputSchema: expect.any(Object), // Now it's a Zod schema object
269 + inputSchema: expect.any(Object),
270 execute: expect.any(Function),
271 });
272 });
ts/scripts/check-provider-compatibility.ts
+23 -2
@@ -48,7 +48,7 @@ const PROVIDER_CONTRACTS: Record<
48 '@composio/llamaindex': {
49 agentic: true,
50 exportName: 'LlamaindexProvider',
51 - verifiedMinimumCore: '0.11.0',
51 + verifiedMinimumCore: '0.14.0',
52 },
53 '@composio/mastra': {
54 agentic: true,
@@ -387,9 +387,30 @@ async function packWorkspaceArtifacts(
387 );
388
389 for (const provider of providers) {
390 + const packedProvider = await packPackage(
391 + provider.directory,
392 + provider.name,
393 + provider.version,
394 + artifactDirectory
395 + );
396 + const providerManifest = await readJsonFile(
397 + path.join(provider.directory, 'package.json'),
398 + PackageManifestSchema
399 + );
400 + // A provider that imports the helper directly needs the workspace build too: the
401 + // registry copy of the same version may predate the entry point it imports.
402 tarballs.set(
403 provider.name,
392 - await packPackage(provider.directory, provider.name, provider.version, artifactDirectory)
404 + providerManifest.dependencies[JSON_SCHEMA_TO_ZOD]
405 + ? await repackCoreTarball(packedProvider, provider.name, {
406 + version: provider.version,
407 + jsonSchemaTarball,
408 + artifactDirectory: path.join(
409 + artifactDirectory,
410 + `provider-${path.basename(provider.directory)}`
411 + ),
412 + })
413 + : packedProvider
414 );
415 }
416