Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.
Mirrored from https://github.com/Camo67/composio.git
Synced Nov 21, 09:45 UTC
fix(anthropic): cap prompt-cache breakpoints at one per request (#4724)
With `cacheTools: true`, the provider put a `cache_control` breakpoint on every wrapped tool and on every `tool_result`. Anthropic allows at most 4 breakpoints per request, shared across the system prompt, tools, and messages. A caller with 5+ tools had the whole request rejected, and a multi-turn loop that appends each turn's tool results would reach the limit again after a few turns. A breakpoint caches everything up to and including it, so one breakpoint on the last tool covers the whole tool list. Fixes #4723 ## Changes - `wrapTools` keeps `cache_control` only on the last tool when `cacheTools` is enabled. The key is deleted from the others, not set to `undefined`, so it is absent from the tool objects. - `handleToolCalls` no longer marks `tool_result` blocks. They are appended to the message history every turn, so a breakpoint on them would add one more to each later request. - `wrapTool` (singular) is unchanged: it still attaches `cache_control` to the single tool it wraps. The fix is scoped to the batch entry point used to build a request's `tools` array. - The README, the constructor JSDoc, and the changeset describe this, including that the cap is per request: lists merged from several `wrapTools` calls each keep a breakpoint on their last tool. - Tests in `test/anthropic.test.ts`: cache only on the last tool of a 3-tool batch (checked with `not.toHaveProperty`), a single-tool batch is still cached, no breakpoint when caching is disabled, and no `tool_result` carries a breakpoint with caching enabled. ## Overlap with #4746 #4746 also fixes #4723 and gives the same result on the tool list and on tool results. This PR additionally removes the key instead of setting it to `undefined`, and updates the README and changeset. ## Type of change - [x] Bug fix - [ ] New feature - [ ] Refactor/Chore - [ ] Documentation - [ ] Breaking change ## How Has This Been Tested? `pnpm vitest run test/anthropic.test.ts`: 22 passed. The new tool_result test fails on the previous code. Run on Node 22.22, the repo declares >=24.17 and prints a warning. `pnpm exec tsc --noEmit` and `pnpm exec oxlint ts/packages/providers/anthropic`: clean. `pnpm exec prettier --check`: clean. ## Checklist - [x] I have read the Code of Conduct and this PR adheres to it - [x] I have read and understand every line of this diff, including any LLM-generated code - [x] Any third-party docs links or promotional content follow the third-party links policy — n/a - [x] I ran linters/tests locally and they passed - [x] I updated documentation as needed (README and JSDoc for `cacheTools`) - [x] I added tests or explain why not applicable - [x] I added a changeset if this change affects published packages --------- Co-authored-by: jkomyno <alberto@composio.dev>