Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.
Mirrored from https://github.com/Camo67/composio.git
Synced Nov 21, 08:30 UTC
fix(json-schema-to-zod): check only base64 `contentEncoding` as base64 (#4910)
## Summary This PR: - supersedes https://github.com/ComposioHQ/composio/pull/4748 by @tarunvashishth, whose fix this keeps (see Credit) - stops the Zod converter (Vercel, LangChain, LlamaIndex, Claude Agent SDK providers) and the Effect converter (`composio execute`) from validating `format: "binary"`, and non-base64 `contentEncoding` values, as base64 - stops telling the model that `format: "binary"` and `contentEncoding: "base16"` fields are `contentEncoding: "base64"` - keeps the base64 check for `contentEncoding: "base64"` (case-insensitive) and leaves `format: "byte"` unchecked, as before - adds Zod/Effect parity cases and a test of the schema the model receives Designated PR reviewer: [@jkomyno](mailto:alberto@composio.dev). Fixes #4747 ## Root cause `parse-string.ts` added Zod's `.base64()` check for `format: "binary"` and for **any** `contentEncoding`. `normalizeFormats` in the Effect converter added `BASE64_PATTERN` under the same condition. Two consequences: 1. **Local rejection.** A model that sent `"hello world"` for a binary field or `"0f"` for a `base16` field was rejected before the tool ran. AI SDK: `AI_InvalidToolInputError`. LangChain: `ToolInputParsingException`. CLI: `ToolInputValidationError`. 2. **Wrong schema sent to the model.** Zod v3's base64 check serializes to `contentEncoding: "base64"`, in the AI SDK's converter and in `zod-to-json-schema` (used by LangChain). So the model was told to base64-encode raw or hex fields. ## Semantics and compatibility decisions - **`format: "binary"`**: OpenAPI 3.0.3 §4.4 defines it as "any sequence of octets"; `byte` is the base64 format. OpenAPI 3.1 §4.8.14.3 states that `format` has no effect on content encoding. It is no longer checked. - **`contentEncoding`**: JSON Schema draft-07 §8.2 makes checking it optional. 2019-09 and 2020-12 (§8.1) make it an annotation only. Its values come from RFC 2045 and RFC 4648, so base16, base32 and quoted-printable aren't base64. - **Explicit base64 stays checked.** The converters already enforced it, and live models comply with it. Removing it would loosen validation that tools may rely on. - **Other encodings are not checked at all.** I didn't add new checks for base16 or base32. - **Encoding names are case-insensitive** (RFC 2045 §6.1), so `BASE64` stays checked, as it was before. - **`format: "binary"` + `contentEncoding: "base64"`**: still checked as base64. `contentEncoding` decides. - **`format: "byte"`**: still unchecked. Checking it would match OpenAPI, but it is new strictness: Zod's base64 check requires padding and would reject unpadded values that work today. - **Python**: `json_schema_to_pydantic_type` already accepts every case here and sends a plain string, so it is unchanged. It never checks explicit base64 either; that difference predates this PR. ## Limitations - **Metadata is lost, not rewritten.** Zod v3 cannot carry `format: "binary"` or a non-base64 `contentEncoding`, so the Zod-based providers now send `{ type: "string" }` for these fields. That drops the annotation, but no longer gives the model a wrong instruction. Keeping the annotation exactly would need the providers to send raw JSON Schema instead of Zod, which is a larger change. - **Explicit base64 still requires padding**, as before. ## Reproduction evidence Synthetic one-property tools whose local executor returns a sentinel. No Composio credentials and no side effects. The Bun scripts are not committed, to keep this PR focused. Each run records three things separately: - the source schema and the schema actually sent, from the captured request body; - the model's raw tool arguments, from the captured response body; - local validation and whether the executor ran. Local validation is judged against the model's actual argument. **Deterministic run.** The real SDK serializes the request; the model reply is canned to return the exact requested argument. - Baseline: 8 of 20 integration runs fail, plus 2 converter failures (binary and base16, in both Zod and Effect). - Fixed: 20 of 20 pass, 0 converter failures. **Live run.** Same inputs and settings for both runs: - models: `gpt-6-luna` and `claude-haiku-5-5`, tool choice forced; - `ai` 7.0.137, `@ai-sdk/openai` 4.0.75, `@ai-sdk/anthropic` 4.0.79, `@langchain/core` 1.2.17, `@langchain/openai` 1.6.2 (Responses API), `@langchain/anthropic` 1.5.12; - each report records which worktree and which built converter ran: baseline zod build `190cc00d4695` (unfixed `next`), fixed build `f7269093a760`. The four paths (Vercel and LangChain, each with OpenAI and Anthropic) gave identical results: | Case (requested input) | Baseline: sent / local / executed | Fixed: sent / local / executed | |---|---|---| | `format: binary` (`hello world`) | `contentEncoding: base64` / rejected / no | `{type: string}` / accepted / yes | | `contentEncoding: base16` (`0f`) | `contentEncoding: base64` / rejected / no | `{type: string}` / accepted / yes | | `contentEncoding: base64` (`aGVsbG8gd29ybGQ=`) | `contentEncoding: base64` / accepted / yes | unchanged | | `contentEncoding: base64` (`not base64!`) | `contentEncoding: base64` / rejected / no | unchanged | | plain string (`hello world`) | `{type: string}` / accepted / yes | unchanged | - Baseline: 12 pass / 8 fail, reproduced exactly in a second run. Fixed: 20 pass / 0 fail. - Both APIs accepted every outgoing schema, so no failure came from an API rejecting a schema. - Both models returned the requested values unchanged in every run. The only behaviour that changed was local validation and the schema sent. ## Changes - `ts/packages/json-schema-to-zod/src/parsers/parse-string.ts`: remove the `binary` base64 case and check `contentEncoding` only when it is base64. - `ts/packages/json-schema-to-effect-schema/src/index.ts`: the same rule for the CLI converter. - `ts/packages/json-schema-to-effect-schema/test/json-schema-to-effect-schema.test.ts`: the parity test asserted the old binary behaviour. It now covers binary text, `byte`, base16, valid and invalid base64, `BASE64`, and binary combined with base64, all through both converters. - `ts/packages/core/test/utils/jsonSchema.test.ts`: the serialized schema keeps `contentEncoding: "base64"` only for base64 fields. - Changeset (`@composio/json-schema-to-zod`, `@composio/json-schema-to-effect-schema`, `@composio/core`, `@composio/slim`: patch) and a `ts/packages/cli/CHANGELOG.md` note, since Changesets ignores the CLI. ## Type of change - [x] Bug fix - [ ] New feature - [ ] Refactor/Chore - [ ] Documentation - [ ] Breaking change ## How Has This Been Tested? - The new parity cases and the new schema-output test fail on unfixed `next` and pass with this change. - `vitest run` passes for: `json-schema-to-zod` (358), `json-schema-to-effect-schema` (156), `core` (1843 + 2 expected fail), `vercel` (15), `langchain` (15), `llamaindex` (22), `claude-agent-sdk` (25). - CLI `tool-input-validation` and `commands/tools` suites: 117 passed. - Other checks: `typecheck` for the three changed packages, `oxlint`, `prettier --check`, `pnpm validate:agent-skills`, `pnpm validate:skill-routing`. ## Checklist - [x] I have read the Code of Conduct and this PR adheres to it - [ ] I have read and understand every line of this diff, including any LLM-generated code - [x] Any third-party docs links or promotional content follow the [third-party links policy](https://github.com/ComposioHQ/composio/blob/next/CONTRIBUTING.md#third-party-links-in-docs) - [x] I ran linters/tests locally and they passed - [x] I updated documentation as needed - [x] I added tests or explain why not applicable - [x] I added a changeset if this change affects published packages ## Credit @tarunvashishth reported #4747 and opened #4748 with the same converter change. This PR keeps that change: the `binary` removal, the base64-only `contentEncoding` check, matching `base64` case-insensitively, and the `0f` base16 case from the #4748 review. What it adds: - live reproduction against OpenAI and Anthropic; - a check of the schema sent to the model; - coverage of `format: "byte"` and of binary combined with base64; - `@composio/json-schema-to-effect-schema` in the changeset. The commit credits them as co-author. #4748 stays open until this PR merges. Co-authored-by: tarunvashishth <tarun.vashishth093@gmail.com>