@devries-cameron20 / composio-1 / commits / a85e79265
Fork 0

Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

fix(docs): page toolkit data through `@composio/client` (#5481)

## Summary Toolkit pages use cursor pagination so Next.js can cache each response separately. The meta-tool reference generates offline from its checked-in schema snapshot. - Live lists of toolkits, tools, and trigger types use `@composio/client` and `@composio/client/pagination`. - `generate:toolkits` follows `next_cursor`, so the next sync can include all 1,208 `microsoft_365` tools. The committed snapshot has 1,000. - Docs - Toolkit Parity checks the 30 largest deployed `.md` pages by production tool count and the 30 largest by snapshot count, plus `/api/toolkits/instant`, every six hours. It opens one tracking issue per outage and sends a Slack alert. - Live toolkit render traversals abort after 15 seconds and fall back to the snapshot. - Oxlint rejects raw `fetch`, `fetcher`, and `fetchWithRetry` calls to `/api/v3*`. - For docs runtime configuration, replace `COMPOSIO_API_BASE` with the host-only `COMPOSIO_BASE_URL`. ## Context Tableau's toolkit grew from 10 to 436 tools. The snapshot has listed all 436 since Nov 13, yet `docs.composio.dev/toolkits/tableau.md` kept listing the old 10 for a week. The Vercel API key was not the cause. It is a valid production key, and a staging key would have answered with a 401, which falls back to the snapshot. The toolkit page and the `.md` route fetched each toolkit's tools in one request. Next.js stores a fetch response in the Data Cache only when the entry is at most 2 MiB, and the entry holds the body base64-encoded, so a response has to stay under about 1.5 MiB. Tableau's response is 3.26 MB. Vercel logged it on every request: ``` Failed to set Next.js data cache for https://backend.composio.dev/api/v3/tools?toolkit_slug=TABLEAU&toolkit_versions=latest&include_pricing=true&limit=1000, items over 2MB can not be cached (4343788 bytes) ``` When that write fails, Next.js keeps serving the entry it already had. Here that was the 10-tool response from before the upgrade, so every background refresh downloaded the new list and then discarded it. The Data Cache outlives deployments, so redeploying changed nothing. The same warning shows up for ACTIVE_CAMPAIGN, TWITTER, WORKDAY and PAGERDUTY. The snapshot generator had a separate cut-off. It asked for `limit=1000` and never read `next_cursor`, so `microsoft_365` (1208 tools) stopped at 1000. Two alternatives don't solve it. - Dropping the fields the pages don't render. `/tools` has no field selection, and the Data Cache stores the response as received. Even reduced to the rendered fields, `microsoft_365` measures 2.25 MB. - Raising the 2 MiB limit. It's fixed in Next.js and only a custom `cacheHandler` lifts it, which would replace Vercel's Data Cache with storage we run. This PR stays within Vercel's design. ## Implications - Live catalog reads and toolkit generation use v3.1. In the original production check, v3 and v3.1 returned byte-identical bodies for tools, trigger types, toolkits, and the changelog. - A large toolkit now takes several requests on a cache miss (18 for Tableau), each cached as its own entry. Runtime list requests retry up to twice, and the whole traversal aborts after 15 seconds. - Tool lists request 25 items per page, toolkit lists 500, and trigger lists 50, which is the cap `/v3.1/triggers_types` documents. - Tableau's largest page is 1.23 MB, a 1.64 MB entry, or 78% of the limit. One Tableau tool accounts for most of that page, so a smaller page size barely helps. If a single tool grows past about 1.5 MiB, no page size can make it cacheable. The parity check alerts at 85%. - The parity run observes up to 60 toolkits, 30 ranked by production count and 30 by snapshot count. Ranking on one side alone let a collapsed snapshot count drop the affected toolkit out of the check, which is the page most worth watching. The second ranking roughly doubles the run's request volume. - The HTML toolkit page now shows triggers for the latest toolkit version, as the `.md` route already did. - Toolkit generation, `/api/toolkits/instant`, and the snapshot-miss lookup use production. Toolkit generation rejects non-production `COMPOSIO_API_BASE` settings. - Meta-tool generation runs without a session or API credentials. Automatic schema refreshes depend on a production Platform HTTPS export configured through the `COMPOSIO_META_TOOLS_SCHEMA_URL` repository variable. Until that export is available, Docs - Update Data renders the reference from the checked-in `public/data/meta-tools.json`. - Meta-tool page titles are escaped into YAML frontmatter, and the schema rejects a multi-line `displayName`, so an export cannot terminate the frontmatter and inject content into the compiled page. - The stale Tableau entry sits under a URL nothing requests anymore, so no cache purge is needed. - `scripts/fetch-openapi.mjs` and `app/api/proxy` still use `fetch`. The first downloads OpenAPI documents, which aren't client operations, and the second forwards whatever request the browser playground sends. ## Review follow-up Six defects turned up in review on the first revision and are fixed on this branch. They change behavior beyond what the original description covered. - A failed page in a paged traversal now fails the generator run. The first revision logged the error, wrote `tools: []`, and exited 0, which would have republished a zero-tool page — the exact failure this PR exists to end. - The parity checker ranked its watch list by the snapshot's own counts. It now ranks on production counts too and unions both, so neither source can shrink the set. - Live render calls passed no abort signal, so a slow API could hold a render through 49 sequential pages. They now bound the traversal at 15 seconds. - Trigger lists were requesting `limit=1000` from an endpoint whose cap is 50. They now page at 50. - The toolkit changelog parsed with an `as` cast and an `Array.isArray` guard. It parses with zod, like every other payload read here. - Meta-tool titles reached YAML frontmatter unescaped while descriptions were escaped. Titles are escaped now, and a multi-line `displayName` is rejected. ## Verification - The production pagination check returned all 1,208 `microsoft_365` tools, 436 Tableau tools, 46 GitHub trigger types, and 1,829 toolkits, including 89 Instant toolkits. - The original live parity check flagged the stale Tableau page (10 of 436 tools) and passed GitHub and Twitter. After the Markdown counter fix, all 50 largest snapshot-backed pages matched their expected counts, including all 370 Datadog MCP tools. - Offline meta-tool generation reproduced the JSON and all seven reference pages byte for byte. Local tests verified HTTPS export imports and rejection of an empty export. - GitHub CI on `f5e3e2e16` is green, including Anti-slop, docs TypeScript validation, link checks, and docs tests. On the same head, the eight touched suites report 94 passing tests, and `tsc --noEmit` and `lint:links` are clean. - The original head `84434a252` passed the same jobs: 825 static and 273 integration docs tests, one integration test skipped. [Anti-slop](https://github.com/ComposioHQ/composio/actions/runs/44587069134/job/135782777866) · [Docs tests](https://github.com/ComposioHQ/composio/actions/runs/44587069403/job/135782778572) - The parity job still reports the committed snapshot as behind production until the next Docs - Update Data run, because `microsoft_365` sits at the 1,000-item cap. The message now names the snapshot rather than blaming the generator.

Alberto Schiabel committed Nov 20, 2026 at 22:32 UTC a85e79265b0f9c036ab57fbfb1f6e4d23099b3e6
29 files changed +1101 -613
.github/workflows/docs-toolkit-parity.yml new
+120
@@ -0,0 +1,120 @@
1 +name: Docs - Toolkit Parity
2 +
3 +# Checks that the deployed toolkit Markdown pages list the tools production
4 +# serves, and that each production tools page stays under the page-size budget
5 +# that keeps it cacheable. A stale Next.js data-cache entry served tableau.md
6 +# with 10 of 436 tools for days without any check failing.
7 +
8 +on:
9 + schedule:
10 + - cron: '23 */6 * * *'
11 + workflow_dispatch:
12 +
13 +permissions:
14 + contents: read
15 +
16 +concurrency:
17 + group: ${{ github.workflow }}
18 + cancel-in-progress: false
19 +
20 +jobs:
21 + parity:
22 + runs-on: ubuntu-latest
23 + permissions:
24 + contents: read
25 + issues: write
26 + defaults:
27 + run:
28 + working-directory: ./docs
29 + env:
30 + # Production read access; the shared COMPOSIO_API_KEY secret is staging-scoped.
31 + COMPOSIO_API_KEY: ${{ secrets.COMPOSIO_DOCS_API_KEY }}
32 +
33 + steps:
34 + - name: Checkout repository
35 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
36 + with:
37 + persist-credentials: false
38 +
39 + - name: Setup Node.js, pnpm, Bun
40 + uses: ./.github/actions/setup-node-pnpm-bun
41 +
42 + - name: Cache bun dependencies
43 + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
44 + with:
45 + path: ~/.bun/install/cache
46 + key: ${{ runner.os }}-bun-${{ hashFiles('docs/bun.lock') }}
47 + restore-keys: |
48 + ${{ runner.os }}-bun-
49 +
50 + - name: Install dependencies
51 + run: bun install --frozen-lockfile --ignore-scripts
52 +
53 + - name: Check toolkit parity
54 + run: |
55 + bun run check:toolkit-parity --markdown "$RUNNER_TEMP/toolkit-parity.md"
56 +
57 + - name: Publish report
58 + if: always()
59 + run: cat "$RUNNER_TEMP/toolkit-parity.md" >> "$GITHUB_STEP_SUMMARY" 2>/dev/null || true
60 +
61 + # One tracking issue per outage, closed by the next clean run, so a
62 + # failing schedule does not re-alert every six hours.
63 + - name: Open tracking issue on failure
64 + id: open-issue
65 + if: failure()
66 + working-directory: ${{ github.workspace }}
67 + env:
68 + GH_TOKEN: ${{ github.token }}
69 + GH_REPO: ${{ github.repository }}
70 + LABEL: docs-toolkit-parity
71 + RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
72 + run: |
73 + gh label create "$LABEL" --color B60205 \
74 + --description "Deployed toolkit pages disagree with production" 2>/dev/null || true
75 +
76 + existing=$(gh issue list --label "$LABEL" --state open --limit 1 --json number --jq '.[0].number // empty')
77 + if [ -n "$existing" ]; then
78 + echo "Issue #$existing is already open; not filing a duplicate."
79 + exit 0
80 + fi
81 +
82 + {
83 + echo "The scheduled \`Docs - Toolkit Parity\` check failed: $RUN_URL"
84 + echo
85 + cat "$RUNNER_TEMP/toolkit-parity.md" 2>/dev/null || echo "The checker exited before writing its report."
86 + echo
87 + echo "A deployed page listing fewer tools than production usually means a Next.js data-cache entry is stale."
88 + echo "Check \`vercel logs --query 'data cache'\` for 'items over 2MB' warnings, and lower the page size in"
89 + echo "\`docs/lib/composio-api.ts\` if a page is over budget. This issue closes after the next clean run."
90 + } > "$RUNNER_TEMP/toolkit-parity-issue.md"
91 +
92 + url=$(gh issue create --title "Deployed toolkit pages disagree with production" \
93 + --label "$LABEL" --body-file "$RUNNER_TEMP/toolkit-parity-issue.md")
94 + echo "issue_url=$url" >> "$GITHUB_OUTPUT"
95 +
96 + - name: Notify Slack
97 + if: failure() && steps.open-issue.outputs.issue_url
98 + uses: slackapi/slack-github-action@dcb1066f776dd043e64d0e8ba94ca15cc7e1875d # v4.0.0
99 + with:
100 + webhook: ${{ secrets.SLACK_POD_DX_WEBHOOK_URL }}
101 + webhook-type: incoming-webhook
102 + payload: |
103 + {"text": "🚨 *Docs Toolkit Parity*: deployed toolkit pages disagree with production. <${{ steps.open-issue.outputs.issue_url }}|Tracking issue>"}
104 +
105 + # GitHub issue cleanup is best effort after a successful parity check.
106 + # ast-grep-ignore: yaml-continue-on-error
107 + - name: Close tracking issue on recovery
108 + if: success()
109 + continue-on-error: true
110 + working-directory: ${{ github.workspace }}
111 + env:
112 + GH_TOKEN: ${{ github.token }}
113 + GH_REPO: ${{ github.repository }}
114 + LABEL: docs-toolkit-parity
115 + run: |
116 + existing=$(gh issue list --label "$LABEL" --state open --limit 1 --json number --jq '.[0].number // empty')
117 + if [ -n "$existing" ]; then
118 + gh issue close "$existing" \
119 + --comment "Toolkit parity recovered in ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}."
120 + fi
.github/workflows/docs-update-data.yml
+4 -2
@@ -82,7 +82,7 @@ jobs:
82 uses: ./.github/actions/setup-node-pnpm-bun
83
84 # Docs must reflect PRODUCTION. No base-URL override is set here: the
85 - # generators default to the production API (docs/scripts/production-api.mjs).
85 + # Toolkit and OpenAPI generators default to production (docs/scripts/production-api.mjs).
86 # Previously this step pointed the fetch at STAGING, which published staging
87 # hosts (and unreleased content) into the committed docs.
88 # NOTE: requires COMPOSIO_DOCS_API_KEY to have PRODUCTION read access.
@@ -108,6 +108,8 @@ jobs:
108 run: bun run generate:api-index
109
110 - name: Generate meta tools reference
111 + env:
112 + COMPOSIO_META_TOOLS_SCHEMA_URL: ${{ vars.COMPOSIO_META_TOOLS_SCHEMA_URL }}
113 run: bun run generate:meta-tools
114
115 # The published KB makes claims about tool slugs, toolkits, and provider docs.
@@ -158,7 +160,7 @@ jobs:
160 - **Toolkit catalog** (`docs/public/data/toolkits.json`, `toolkits-list.json`) — refreshed list of available toolkits, auth schemes, and tools from the backend API
161 - **OpenAPI specs** (`docs/public/openapi.json`, `docs/public/openapi-v3.json`, `docs/public/openapi-webhooks.json`) — latest v3.1 and v3.0 API specifications plus the webhook-events spec, fetched from production
162 - **API reference pages** (`docs/content/reference/api-reference/`, `docs/content/reference/v3/api-reference/`) — regenerated index pages for both API versions
161 - - **Meta tools reference** (`docs/public/data/meta-tools.json`, `docs/content/toolkits/meta-tools/*.mdx`) — updated meta tool schemas and reference docs
163 + - **Meta tools reference** (`docs/public/data/meta-tools.json`, `docs/content/toolkits/meta-tools/*.mdx`) — generated from the bundled schema snapshot or the configured Platform export
164 branch: docs/auto-update-data
165 base: next
166 add-paths: |
docs/.oxlintrc.json
+4
@@ -67,6 +67,10 @@
67 {
68 "selector": "TemplateElement[value.raw=/dashboard\\.composio\\.dev\\u002F(?!~\\u002F(project|org)([\\u002F?#]|$)|login([\\u002F?#]|$))/]",
69 "message": "Composio dashboard links with a path must be go-links (/~/project/... or /~/org/...) or /login so they resolve inside the user workspace."
70 + },
71 + {
72 + "selector": "CallExpression[callee.name=/^(fetch|fetcher|fetchWithRetry)$/] :matches(TemplateElement[value.raw=/\\/api\\/v3/], Literal[value=/\\/api\\/v3/], Identifier[name=/^(PRODUCTION_API_V3_URL|PRODUCTION_API_V31_URL|API_BASE)$/])",
73 + "message": "Call the Composio API through @composio/client: createComposioClient and the list helpers in lib/composio-api.ts. They page every list, so no single response exceeds the 2 MB Next.js data-cache entry limit, which keeps serving a stale copy."
74 }
75 ]
76 },
docs/app/(home)/toolkits/[[...slug]]/page.tsx
+18 -50
@@ -16,42 +16,31 @@ import type { Metadata } from 'next';
16 import type { Tool, Trigger } from '@/types/toolkit';
17 import type { FaqItem } from '@/components/toolkits/faq-section';
18 import { apiToolListSchema, apiTriggerListSchema } from '@/lib/toolkit-schema';
19 -
20 -const API_BASE = process.env.COMPOSIO_API_BASE || 'https://backend.composio.dev/api/v3';
21 -const API_KEY = process.env.COMPOSIO_API_KEY;
19 +import {
20 + createComposioClientFromEnv,
21 + LIVE_TRAVERSAL_TIMEOUT_MS,
22 + listToolkitTools,
23 + listToolkitTriggers,
24 +} from '@/lib/composio-api';
25 +
26 +// COMPOSIO_BASE_URL selects the API host; production by default.
27 +const client = process.env.COMPOSIO_API_KEY
28 + ? createComposioClientFromEnv(process.env, { revalidate: 3600 })
29 + : null;
30
31 // Snapshot misses are resolved from production on demand.
32 export const dynamicParams = true;
33
34 // Fetch detailed tool info from Composio API (server-side only)
35 // Returns null on failure, empty array if toolkit has no tools
28 -async function fetchDetailedTools(
29 - toolkitSlug: string
30 -): Promise<Tool[] | null> {
31 - if (!API_KEY) {
36 +async function fetchDetailedTools(toolkitSlug: string): Promise<Tool[] | null> {
37 + if (!client) {
38 console.warn('[Toolkits] COMPOSIO_API_KEY not set, skipping detailed tool fetch');
39 return null;
40 }
41
42 try {
37 - const response = await fetch(
38 - `${API_BASE}/tools?toolkit_slug=${toolkitSlug.toUpperCase()}&toolkit_versions=latest&include_pricing=true&limit=10000`,
39 - {
40 - headers: {
41 - 'Content-Type': 'application/json',
42 - 'x-api-key': API_KEY,
43 - },
44 - next: { revalidate: 3600 },
45 - }
46 - );
47 -
48 - if (!response.ok) {
49 - console.warn(`[Toolkits] Failed to fetch tools for ${toolkitSlug}: ${response.status}`);
50 - return null;
51 - }
52 -
53 - const parsed = apiToolListSchema.safeParse(await response.json());
54 - return parsed.success ? parsed.data : [];
43 + return apiToolListSchema.parse(await listToolkitTools(client, toolkitSlug, { signal: AbortSignal.timeout(LIVE_TRAVERSAL_TIMEOUT_MS) }));
44 } catch (error) {
45 console.error(`[Toolkits] Error fetching detailed tools for ${toolkitSlug}:`, error);
46 return null;
@@ -60,41 +49,20 @@ async function fetchDetailedTools(
49
50 // Fetch detailed trigger info from Composio API (server-side only)
51 // Returns null on failure, empty array if toolkit has no triggers
63 -async function fetchDetailedTriggers(
64 - toolkitSlug: string,
65 - version?: string | null
66 -): Promise<Trigger[] | null> {
67 - if (!API_KEY) {
52 +async function fetchDetailedTriggers(toolkitSlug: string): Promise<Trigger[] | null> {
53 + if (!client) {
54 console.warn('[Toolkits] COMPOSIO_API_KEY not set, skipping detailed trigger fetch');
55 return null;
56 }
57
58 try {
73 - const response = await fetch(
74 - `${API_BASE}/triggers_types?toolkit_slugs=${toolkitSlug.toUpperCase()}&toolkit_versions=latest&limit=10000${version ? `&version=${encodeURIComponent(version)}` : ''}`,
75 - {
76 - headers: {
77 - 'Content-Type': 'application/json',
78 - 'x-api-key': API_KEY,
79 - },
80 - next: { revalidate: 3600 }, // Cache for 1 hour
81 - }
82 - );
83 -
84 - if (!response.ok) {
85 - console.warn(`[Toolkits] Failed to fetch triggers for ${toolkitSlug}: ${response.status}`);
86 - return null;
87 - }
88 -
89 - const parsed = apiTriggerListSchema.safeParse(await response.json());
90 - return parsed.success ? parsed.data : [];
59 + return apiTriggerListSchema.parse(await listToolkitTriggers(client, toolkitSlug, { signal: AbortSignal.timeout(LIVE_TRAVERSAL_TIMEOUT_MS) }));
60 } catch (error) {
61 console.error(`[Toolkits] Error fetching detailed triggers for ${toolkitSlug}:`, error);
62 return null;
63 }
64 }
65
97 -
66 async function readToolkitFaq(slug: string): Promise<FaqItem[] | null> {
67 const filePath = join(process.cwd(), 'content/toolkits/faq', `${slug}.md`);
68 let content: string;
@@ -225,7 +193,7 @@ export default async function ToolkitsPage({ params }: { params: Promise<{ slug?
193 // Fetch detailed tool/trigger info and FAQ content in parallel
194 const [detailedTools, detailedTriggers, faq] = await Promise.all([
195 fetchDetailedTools(toolkit.slug),
228 - fetchDetailedTriggers(toolkit.slug, toolkit.version),
196 + fetchDetailedTriggers(toolkit.slug),
197 readToolkitFaq(toolkit.slug),
198 ]);
199
docs/app/api/tools/[slug]/route.ts
+16 -20
@@ -1,6 +1,8 @@
1 import { NextRequest, NextResponse } from 'next/server';
2 import { z } from 'zod';
3 +import { APIError } from '@composio/client';
4 import { instantSchema } from '@/lib/instant';
5 +import { createComposioClientFromEnv } from '@/lib/composio-api';
6
7 const toolDetailsSchema = z.object({
8 input_parameters: z.unknown().optional(),
@@ -10,8 +12,10 @@ const toolDetailsSchema = z.object({
12 instant: instantSchema,
13 });
14
13 -const API_BASE = process.env.COMPOSIO_API_BASE || 'https://backend.composio.dev/api/v3';
14 -const API_KEY = process.env.COMPOSIO_API_KEY;
15 +// COMPOSIO_BASE_URL selects the API host; production by default.
16 +const client = process.env.COMPOSIO_API_KEY
17 + ? createComposioClientFromEnv(process.env, { revalidate: 3600 })
18 + : null;
19
20 export async function GET(
21 request: NextRequest,
@@ -20,7 +24,7 @@ export async function GET(
24 const { slug } = await params;
25 const version = request.nextUrl.searchParams.get('version') || 'latest';
26
23 - if (!API_KEY) {
27 + if (!client) {
28 return NextResponse.json({ error: 'API key not configured' }, { status: 500 });
29 }
30
@@ -29,22 +33,11 @@ export async function GET(
33 }
34
35 try {
32 - const response = await fetch(
33 - `${API_BASE}/tools/${slug.toUpperCase()}?version=${encodeURIComponent(version)}${version === 'latest' ? '&include_pricing=true' : ''}`,
34 - {
35 - headers: {
36 - 'Content-Type': 'application/json',
37 - 'x-api-key': API_KEY,
38 - },
39 - next: { revalidate: 3600 },
40 - }
41 - );
42 -
43 - if (!response.ok) {
44 - return NextResponse.json({ error: 'Tool not found' }, { status: response.status });
45 - }
46 -
47 - const tool = toolDetailsSchema.parse(await response.json());
36 + const response = await client.tools.retrieve(slug.toUpperCase(), {
37 + version,
38 + ...(version === 'latest' ? { include_pricing: true } : {}),
39 + });
40 + const tool = toolDetailsSchema.parse(response);
41
42 const inputSchema = tool.input_parameters || tool.parameters;
43 const outputSchema = tool.output_parameters || tool.response;
@@ -54,7 +47,10 @@ export async function GET(
47 output_parameters: outputSchema || null,
48 ...(version === 'latest' && tool.instant ? { instant: tool.instant } : {}),
49 });
57 - } catch {
50 + } catch (error) {
51 + if (error instanceof APIError && error.status !== undefined && error.status < 500) {
52 + return NextResponse.json({ error: 'Tool not found' }, { status: error.status });
53 + }
54 return NextResponse.json({ error: 'Failed to fetch tool' }, { status: 500 });
55 }
56 }
docs/app/llms.mdx/[[...slug]]/route.ts
+18 -67
@@ -30,6 +30,12 @@ import { encodeMarkdownTableCell } from '@/lib/markdown-escaping';
30 import type { MetaTool, MetaToolParameter } from '@/lib/meta-tools-data';
31 import type { Toolkit, Tool, Trigger, ParameterSchema } from '@/types/toolkit';
32 import { apiToolListSchema, apiTriggerListSchema } from '@/lib/toolkit-schema';
33 +import {
34 + createComposioClientFromEnv,
35 + LIVE_TRAVERSAL_TIMEOUT_MS,
36 + listToolkitTools,
37 + listToolkitTriggers,
38 +} from '@/lib/composio-api';
39 import { z } from 'zod';
40 import {
41 getKnowledgeByProductArea,
@@ -45,90 +51,35 @@ import {
51
52 export const revalidate = false;
53
48 -const API_BASE = process.env.COMPOSIO_API_BASE || 'https://backend.composio.dev/api/v3';
49 -const API_KEY = process.env.COMPOSIO_API_KEY;
50 -const API_FETCH_LIMIT = 1000; // Note: Toolkits with more items will be truncated
54 +// COMPOSIO_BASE_URL selects the API host; production by default.
55 +const client = process.env.COMPOSIO_API_KEY
56 + ? createComposioClientFromEnv(process.env, { revalidate: 3600 })
57 + : null;
58
59 // Fetch detailed tool info from Composio API
60 async function fetchDetailedTools(toolkitSlug: string): Promise<Tool[] | null> {
54 - if (!API_KEY) {
61 + if (!client) {
62 return null;
63 }
64
65 try {
59 - const response = await fetch(
60 - `${API_BASE}/tools?toolkit_slug=${toolkitSlug.toUpperCase()}&toolkit_versions=latest&include_pricing=true&limit=${API_FETCH_LIMIT}`,
61 - {
62 - headers: {
63 - 'Content-Type': 'application/json',
64 - 'x-api-key': API_KEY,
65 - },
66 - next: { revalidate: 3600 },
67 - }
68 - );
69 -
70 - if (!response.ok) {
71 - console.warn(`[LLM Markdown] Failed to fetch tools for ${toolkitSlug}: ${response.status}`);
72 - return null;
73 - }
74 -
75 - const parsed = apiToolListSchema.safeParse(await response.json());
76 - if (!parsed.success) {
77 - console.warn(`[LLM Markdown] Invalid API response format for toolkit ${toolkitSlug}`);
78 - return null;
79 - }
80 -
81 - if (parsed.data.length >= API_FETCH_LIMIT) {
82 - console.warn(
83 - `[LLM Markdown] Toolkit ${toolkitSlug} has ${parsed.data.length}+ tools, results may be truncated`
84 - );
85 - }
86 -
87 - return parsed.data;
88 - } catch {
66 + return apiToolListSchema.parse(await listToolkitTools(client, toolkitSlug, { signal: AbortSignal.timeout(LIVE_TRAVERSAL_TIMEOUT_MS) }));
67 + } catch (error) {
68 + console.warn(`[LLM Markdown] Failed to fetch tools for ${toolkitSlug}:`, error);
69 return null;
70 }
71 }
72
73 // Fetch detailed trigger info from Composio API
74 async function fetchDetailedTriggers(toolkitSlug: string): Promise<Trigger[] | null> {
95 - if (!API_KEY) {
75 + if (!client) {
76 return null;
77 }
78
79 try {
100 - const response = await fetch(
101 - `${API_BASE}/triggers_types?toolkit_slugs=${toolkitSlug.toUpperCase()}&toolkit_versions=latest&limit=${API_FETCH_LIMIT}`,
102 - {
103 - headers: {
104 - 'Content-Type': 'application/json',
105 - 'x-api-key': API_KEY,
106 - },
107 - next: { revalidate: 3600 },
108 - }
109 - );
110 -
111 - if (!response.ok) {
112 - console.warn(
113 - `[LLM Markdown] Failed to fetch triggers for ${toolkitSlug}: ${response.status}`
114 - );
115 - return null;
116 - }
117 -
118 - const parsed = apiTriggerListSchema.safeParse(await response.json());
119 - if (!parsed.success) {
120 - console.warn(`[LLM Markdown] Invalid API response format for triggers ${toolkitSlug}`);
121 - return null;
122 - }
123 -
124 - if (parsed.data.length >= API_FETCH_LIMIT) {
125 - console.warn(
126 - `[LLM Markdown] Toolkit ${toolkitSlug} has ${parsed.data.length}+ triggers, results may be truncated`
127 - );
128 - }
129 -
130 - return parsed.data;
131 - } catch {
80 + return apiTriggerListSchema.parse(await listToolkitTriggers(client, toolkitSlug, { signal: AbortSignal.timeout(LIVE_TRAVERSAL_TIMEOUT_MS) }));
81 + } catch (error) {
82 + console.warn(`[LLM Markdown] Failed to fetch triggers for ${toolkitSlug}:`, error);
83 return null;
84 }
85 }
docs/bun.lock
+1
@@ -47,6 +47,7 @@
47 "@anthropic-ai/sdk": "^0.115.0",
48 "@composio/anthropic": "0.11.8",
49 "@composio/claude-agent-sdk": "0.13.1",
50 + "@composio/client": "2.1.33",
51 "@composio/core": "0.30.0",
52 "@composio/google": "0.13.0",
53 "@composio/langchain": "0.11.3",
docs/decisions/README.md
+2 -1
@@ -10,7 +10,8 @@ Read these when a docs change touches an existing architecture, generated-data p
10 - `public-knowledge-base.md` - support-portal KB publication, unified search,
11 and Composio For You snapshot rules.
12 - `toolkit-page-availability.md` - docs-sync permissions, optional KB isolation,
13 - and snapshot-miss toolkit page fallback.
13 + snapshot-miss toolkit page fallback, and paginated live tool lists with their
14 + parity monitor.
15 - `toolkits.md` - toolkit page data, rendering, and live Instant eligibility filter decisions.
16 - `cookbooks-revamp-plan.md` - historical cookbook revamp tracker.
17 - `apps-and-toolkits.md` - Apps, Composio Built toolkits, and Provider MCP toolkits: vocabulary, sibling rendering, and search-contract wording.
docs/decisions/toolkit-page-availability.md
+35 -16
@@ -48,24 +48,43 @@ toolkit, OpenAPI, API index, or meta-tool generation.
48 to a value other than `0`, then deploy again.
49 - The guaranteed code rollback is to revert the toolkit route integration and
50 deploy the reverted revision.
51 -
52 -## Verification
53 -
54 -A local workflow test cannot prove GitHub App installation permissions. The
55 -required proof is a `workflow_dispatch` run at the exact reviewed head that
56 -passes the token step and reaches `Generate toolkits data`.
57 -
58 -Run `32797593048` at commit `3716bda` passed both the token step and
59 -`Generate toolkits data`. This proves the installation accepted the token
60 -request at that exact head. The hosted runtime contains a `COMPOSIO_API_KEY`
61 -environment variable name, but its production scope has not been verified.
62 -Production fallback behavior therefore remains unverified until a deployed
63 -snapshot-miss toolkit page returns 200 with the expected toolkit data.
64 -
65 -For future changes:
66 -
51 +- Every docs catalog read (toolkits, tools, trigger types, and the toolkit
52 + changelog) goes through `@composio/client` (v3.1), built in
53 + `lib/composio-api.ts`. `scripts/fetch-openapi.mjs` and `app/api/proxy` keep
54 + raw `fetch`: the first downloads OpenAPI documents, and the second forwards
55 + arbitrary playground requests. The toolkit page, `/llms.mdx` and
56 + `/api/tools/[slug]` use `Composio.fromEnv`, so `COMPOSIO_BASE_URL` selects
57 + the API host; the toolkit generator, `/api/toolkits/instant` and the snapshot-miss
58 + lookup always call production. Lists use its
59 + helpers, which page with `@composio/client/pagination`: toolkit pages and
60 + `/llms.mdx/toolkits/<slug>` at `TOOL_PAGE_SIZE`, `/api/toolkits/instant` at
61 + `TOOLKIT_PAGE_SIZE`, and `generate:toolkits` at the API's per-endpoint cap
62 + (1000 tools or toolkits, 50 trigger types). Live traversals during a page
63 + render abort after `LIVE_TRAVERSAL_TIMEOUT_MS` and fall back to the snapshot.
64 + A failed traversal fails `generate:toolkits` instead of publishing an empty
65 + list. In
66 + Next.js each page is its own Data Cache entry. Next.js does not cache an
67 + entry over 2 MiB, and keeps serving the previous entry; an entry holds the body
68 + base64-encoded, so a response must stay under about 1.5 MiB. The limit is
69 + fixed in Next.js and lifted only by a custom `cacheHandler`. A single
70 + unpaginated request froze `tableau.md` at 10 of 436 tools after the toolkit
71 + grew, and the snapshot stopped `microsoft_365` at 1000 of 1208 tools. oxlint
72 + rejects a raw `fetch`, `fetcher` or `fetchWithRetry` call to `/api/v3*`.
73 +- Dropping response fields does not replace paging. The API has no field
74 + selection and the Data Cache stores response bodies as received. Even reduced
75 + to the fields the pages render, `microsoft_365` measured 2.25 MB.
76 - Run `bun run lint:links` from `docs/` after editing this record.
77 - Dispatch `Docs - Update Data` at the exact reviewed head after changing token
78 permissions.
79 - After deployment, verify one snapshot-backed slug and one valid production
80 slug absent from the committed snapshot both return 200.
81 +- `Docs - Toolkit Parity` runs `bun run check:toolkit-parity` every six hours.
82 + For the toolkits with the most tools, ranked by production count and by
83 + snapshot count, it fails when a deployed `.md` page
84 + lists a tool count matching neither production nor the snapshot, when the
85 + snapshot stops at the 1000-item page cap below production, when
86 + `/api/toolkits/instant` differs from production, or when a production
87 + `/tools` or `/toolkits` page would make a cache entry over
88 + `PAGE_BYTES_BUDGET` (85% of 2 MiB). A failure opens one
89 + `docs-toolkit-parity` issue, posts it to Slack, and the next clean run
90 + closes it.
docs/lib/composio-api.ts new
+123
@@ -0,0 +1,123 @@
1 +import { Auth, Composio, createFetchTransport, type ClientOptions } from '@composio/client';
2 +import { paginate } from '@composio/client/pagination';
3 +
4 +/**
5 + * Tools per page. Next.js does not cache an entry over 2 MiB, and when that
6 + * write fails it keeps serving the previously cached response. An entry holds
7 + * the body base64-encoded, so a response must stay under about 1.5 MiB. At 25
8 + * the largest production tools page measured 1.23 MB (Tableau, a 1.64 MB
9 + * entry); one Tableau tool dominates it, so a smaller page barely helps.
10 + */
11 +export const TOOL_PAGE_SIZE = 25;
12 +
13 +/** Toolkits per page. At 500 the largest production page measured 404 KB. */
14 +export const TOOLKIT_PAGE_SIZE = 500;
15 +
16 +/** The most items one `/tools` or `/toolkits` list request returns; the API caps `limit` here. */
17 +export const API_PAGE_LIMIT = 1000;
18 +
19 +/** The most items one `/triggers_types` list request returns; the API caps `limit` here. */
20 +export const TRIGGER_PAGE_LIMIT = 50;
21 +
22 +/**
23 + * Milliseconds a live list traversal may take during a page render. A larger
24 + * toolkit takes many sequential pages, so the bound covers the whole traversal;
25 + * when it fires the caller falls back to the committed snapshot.
26 + */
27 +export const LIVE_TRAVERSAL_TIMEOUT_MS = 15_000;
28 +
29 +type Fetch = (input: RequestInfo | URL, init?: RequestInit) => Promise<Response>;
30 +
31 +export interface ComposioClientOptions {
32 + /**
33 + * Seconds a response stays fresh in the Next.js Data Cache. Leave it unset
34 + * outside Next.js; each page of a list is cached as its own entry.
35 + */
36 + revalidate?: number;
37 + /** Retries for idempotent requests; the client defaults to 2. */
38 + maxRetries?: number;
39 + /** Milliseconds one request may take, including a custom `fetch`'s own retries. */
40 + timeout?: number;
41 + fetch?: Fetch;
42 +}
43 +
44 +function clientOverrides({ revalidate, maxRetries, timeout, fetch }: ComposioClientOptions) {
45 + return {
46 + transport: createFetchTransport({
47 + fetch,
48 + defaults: revalidate === undefined ? undefined : { next: { revalidate } },
49 + }),
50 + ...(maxRetries === undefined ? {} : { maxRetries }),
51 + ...(timeout === undefined ? {} : { timeout }),
52 + } satisfies Partial<ClientOptions>;
53 +}
54 +
55 +/**
56 + * A production API client authenticated with a project API key. For callers
57 + * that must reach production whatever the environment says: the generators,
58 + * the Instant toolkit list, and the snapshot-miss toolkit lookup.
59 + */
60 +export function createComposioClient(apiKey: string, options: ComposioClientOptions = {}): Composio {
61 + return new Composio({ auth: Auth.project(apiKey), ...clientOverrides(options) });
62 +}
63 +
64 +/**
65 + * A client configured by `Composio.fromEnv`: credentials from `COMPOSIO_API_KEY`
66 + * and the API host from `COMPOSIO_BASE_URL`, defaulting to production.
67 + */
68 +export function createComposioClientFromEnv(
69 + environment: Readonly<Record<string, string | undefined>>,
70 + options: ComposioClientOptions = {}
71 +): Composio {
72 + return Composio.fromEnv(environment, clientOverrides(options));
73 +}
74 +
75 +interface ListOptions {
76 + pageSize?: number;
77 + /** Aborts the whole traversal, not one page. */
78 + signal?: AbortSignal;
79 +}
80 +
81 +/** Every tool of a toolkit's latest version, with pricing. */
82 +export function listToolkitTools(
83 + client: Composio,
84 + toolkitSlug: string,
85 + { pageSize = TOOL_PAGE_SIZE, signal }: ListOptions = {}
86 +) {
87 + return paginate(({ cursor }) =>
88 + client.tools.list(
89 + {
90 + toolkit_slug: toolkitSlug.toUpperCase(),
91 + toolkit_versions: 'latest',
92 + include_pricing: true,
93 + limit: pageSize,
94 + cursor,
95 + },
96 + { signal }
97 + )
98 + ).toArray();
99 +}
100 +
101 +/** Every trigger type of a toolkit's latest version. */
102 +export function listToolkitTriggers(
103 + client: Composio,
104 + toolkitSlug: string,
105 + { pageSize = TRIGGER_PAGE_LIMIT, signal }: ListOptions = {}
106 +) {
107 + return paginate(({ cursor }) =>
108 + client.triggersTypes.list(
109 + { toolkit_slugs: [toolkitSlug.toUpperCase()], toolkit_versions: 'latest', limit: pageSize, cursor },
110 + { signal }
111 + )
112 + ).toArray();
113 +}
114 +
115 +/** Every toolkit in the catalog. Instant eligibility arrives without `include_pricing`. */
116 +export function listToolkits(
117 + client: Composio,
118 + { pageSize = TOOLKIT_PAGE_SIZE, signal }: ListOptions = {}
119 +) {
120 + return paginate(({ cursor }) =>
121 + client.toolkits.list({ limit: pageSize, cursor }, { signal })
122 + ).toArray();
123 +}
docs/lib/instant-toolkit-api.ts
+23 -38
@@ -1,44 +1,29 @@
1 import { z } from 'zod';
2 -import { PRODUCTION_API_V3_URL } from '../scripts/production-api.mjs';
2 +import { createComposioClient, listToolkits } from './composio-api';
3 import { isPublicToolkitSlug } from './public-toolkit-policy';
4
5 -const pageSchema = z.object({
6 - items: z.array(z.object({
7 - slug: z.string().min(1),
8 - instant: z.object({ supported: z.boolean() }).nullish(),
9 - })),
10 - next_cursor: z.string().nullish(),
11 -});
5 +const toolkitsSchema = z.array(z.object({
6 + slug: z.string().min(1),
7 + instant: z.object({ supported: z.boolean() }).nullish(),
8 +}));
9 +
10 +/**
11 + * Reduces a complete /toolkits listing to its public Instant-supported slugs.
12 + * Throws when no toolkit carries eligibility, which means the listing is not
13 + * eligibility-aware rather than that no toolkit qualifies.
14 + */
15 +export function instantToolkitSlugs(items: unknown[]): string[] {
16 + const toolkits = toolkitsSchema.parse(items);
17 + if (!toolkits.some(toolkit => toolkit.instant != null)) throw new Error('Toolkit eligibility missing');
18 + const slugs = toolkits
19 + .filter(toolkit => toolkit.instant?.supported === true)
20 + .map(toolkit => toolkit.slug.toLowerCase())
21 + .filter(isPublicToolkitSlug);
22 + return [...new Set(slugs)];
23 +}
24
25 // Called only by the server route. Never return partial pages as a complete list.
14 -export async function fetchInstantToolkitSlugs(apiKey: string, fetcher: typeof fetch = fetch): Promise<string[]> {
15 - const slugs = new Set<string>();
16 - const cursors = new Set<string>();
17 - let cursor: string | undefined;
18 - let hasEligibility = false;
19 - const signal = AbortSignal.timeout(15000);
20 - for (let page = 0; page < 12; page++) {
21 - const params = new URLSearchParams({ limit: '1000', include_pricing: 'true' });
22 - if (cursor) params.set('cursor', cursor);
23 - const response = await fetcher(`${PRODUCTION_API_V3_URL}/toolkits?${params}`, {
24 - headers: { 'x-api-key': apiKey },
25 - next: { revalidate: 300 },
26 - signal,
27 - });
28 - if (!response.ok) throw new Error('Toolkit eligibility unavailable');
29 - const data = pageSchema.parse(await response.json());
30 - for (const toolkit of data.items) {
31 - if (toolkit.instant != null) hasEligibility = true;
32 - const slug = toolkit.slug.toLowerCase();
33 - if (toolkit.instant?.supported === true && isPublicToolkitSlug(slug)) slugs.add(slug);
34 - }
35 - if (!data.next_cursor) {
36 - if (!hasEligibility) throw new Error('Toolkit eligibility missing');
37 - return [...slugs];
38 - }
39 - if (cursors.has(data.next_cursor)) throw new Error('Repeated toolkit cursor');
40 - cursors.add(data.next_cursor);
41 - cursor = data.next_cursor;
42 - }
43 - throw new Error('Toolkit eligibility pagination incomplete');
26 +export async function fetchInstantToolkitSlugs(apiKey: string, fetcher?: typeof fetch): Promise<string[]> {
27 + const client = createComposioClient(apiKey, { revalidate: 300, fetch: fetcher });
28 + return instantToolkitSlugs(await listToolkits(client, { signal: AbortSignal.timeout(15000) }));
29 }
docs/lib/meta-tool-overrides.ts
+2 -2
@@ -4,9 +4,9 @@ import { resolveToolkitCount } from './toolkit-count';
4 * Hand-written copy overrides for meta tool reference pages.
5 *
6 * The meta tool JSON (`public/data/meta-tools.json`) and the generated MDX are
7 - * regenerated from the Tool Router API by `scripts/generate-meta-tools.ts`, so
7 + * generated by `scripts/generate-meta-tools.ts`, so
8 * editing those gets clobbered. This map survives regeneration: it is merged
9 - * over the API data at read time in `lib/meta-tools-data.ts`.
9 + * over the schema snapshot at read time in `lib/meta-tools-data.ts`.
10 *
11 * Each entry provides Modal-voice display copy. The API still owns the input
12 * and response schemas; we only override the prose.
docs/lib/meta-tools-data.ts
+13 -2
@@ -87,7 +87,10 @@ export const metaToolSchema = z
87 .object({
88 slug: z.string(),
89 name: z.string(),
90 - displayName: z.string(),
90 + // Generated pages put it on one frontmatter line.
91 + displayName: z.string().refine(value => !/[\r\n]/.test(value), {
92 + message: 'Meta-tool displayName must be a single line',
93 + }),
94 description: z.string(),
95 tags: z.array(z.string()),
96 inputParameters: parameterSchema.transform(toDisplayParameter),
@@ -105,7 +108,15 @@ export const metaToolSchema = z
108
109 export type MetaTool = z.infer<typeof metaToolSchema>;
110
108 -const metaToolsSchema = z.array(metaToolSchema);
111 +export const metaToolsSchema = z
112 + .array(metaToolSchema)
113 + .min(1)
114 + .refine(tools => tools.every(tool => /^COMPOSIO_[A-Z0-9_]+$/.test(tool.slug)), {
115 + message: 'Invalid meta-tool slug',
116 + })
117 + .refine(tools => new Set(tools.map(tool => tool.slug)).size === tools.length, {
118 + message: 'Meta-tool slugs must be unique',
119 + });
120 const META_TOOLS_PATH = join(process.cwd(), 'public/data/meta-tools.json');
121
122 let cached: { list: MetaTool[]; bySlug: Map<string, MetaTool> } | null = null;
docs/lib/toolkit-api.ts
+4 -17
@@ -1,7 +1,7 @@
1 import { z } from 'zod';
2 import { instantSchema } from './instant';
3 import { isPublicToolkitSlug, normalizeToolkitSlug } from '@/lib/public-toolkit-policy';
4 -import { PRODUCTION_API_V3_URL } from '@/scripts/production-api.mjs';
4 +import { createComposioClient } from './composio-api';
5 import type { AuthConfigDetail, AuthConfigField, Toolkit } from '@/types/toolkit';
6
7 const NEGATIVE_CACHE_TTL_MS = 60_000;
@@ -261,24 +261,11 @@ export async function fetchToolkitFromProduction(slug: string): Promise<Toolkit
261
262 async function fetchAndMapToolkit(normalizedSlug: string, apiKey: string): Promise<Toolkit | null> {
263 try {
264 - const response = await fetch(`${PRODUCTION_API_V3_URL}/toolkits/${normalizedSlug}`, {
265 - headers: {
266 - 'Content-Type': 'application/json',
267 - 'x-api-key': apiKey,
268 - },
269 - next: { revalidate: 3600 },
264 + const client = createComposioClient(apiKey, { revalidate: 3600, maxRetries: 0 });
265 + const response = await client.toolkits.retrieve(normalizedSlug, null, {
266 signal: AbortSignal.timeout(PRODUCTION_REQUEST_TIMEOUT_MS),
267 });
272 -
273 - if (!response.ok) {
274 - recordNegative(normalizedSlug);
275 - console.warn(
276 - `[Toolkits] Production toolkit lookup failed for "${normalizedSlug}": ${response.status}`
277 - );
278 - return null;
279 - }
280 -
281 - const payload = rawToolkitSchema.parse(await response.json());
268 + const payload = rawToolkitSchema.parse(response);
269 if (payload.type !== 'native' || payload.enabled !== true) {
270 recordNegative(normalizedSlug);
271 console.warn(
docs/package.json
+2
@@ -32,6 +32,7 @@
32 "test:kb-publish": "bun tests/e2e/kb-publish.ts",
33 "generate:toolkits": "bun scripts/generate-toolkits.ts",
34 "generate:toolkit-versions": "bun scripts/refresh-toolkit-versions.ts",
35 + "check:toolkit-parity": "bun scripts/check-toolkit-parity.ts",
36 "generate:meta-tools": "bun scripts/generate-meta-tools.ts",
37 "generate:api-index": "bun scripts/generate-api-index.ts",
38 "check:docs-parity": "bun scripts/check-docs-parity.ts",
@@ -88,6 +89,7 @@
89 "@anthropic-ai/sdk": "^0.115.0",
90 "@composio/anthropic": "0.11.8",
91 "@composio/claude-agent-sdk": "0.13.1",
92 + "@composio/client": "2.1.33",
93 "@composio/core": "0.30.0",
94 "@composio/google": "0.13.0",
95 "@composio/langchain": "0.11.3",
docs/scripts/README.md
+11 -4
@@ -2,8 +2,8 @@
2
3 ## fetch-with-retry.ts
4
5 -Shared `fetchWithRetry` helper used by the data-generation scripts
6 -(`generate-toolkits.ts`, `generate-meta-tools.ts`). It wraps the global `fetch`
5 +Shared `fetchWithRetry` helper used by the toolkit generator and parity checker.
6 +It wraps the global `fetch`
7 with rate-limit-aware retry/backoff: it retries `429` and transient 5xx
8 responses, honors the `Retry-After` header when present, otherwise falls back to
9 exponential backoff with jitter, and caps attempts so CI still fails fast when
@@ -12,8 +12,15 @@ the backend is genuinely down.
12 This matters because `generate-toolkits.ts` issues ~6500 requests per run
13 (a few catalog pages + 3 per toolkit across a ~2.1k catalog), which can exceed
14 the backend request limit.
15 -Before this helper, runs failed with `429`, and `generate-meta-tools.ts` — which
16 -runs immediately after — inherited the exhausted rate-limit window.
15 +
16 +## Meta-tools reference
17 +
18 +`bun run generate:meta-tools` renders the reference from
19 +`public/data/meta-tools.json`. Set `COMPOSIO_META_TOOLS_SCHEMA_URL` to the
20 +production Platform HTTPS export to update the snapshot before rendering. The
21 +export must return the same JSON array format as the snapshot. Docs - Update Data
22 +reads the URL from the repository variable of the same name. With the variable
23 +unset, reference pages use the checked-in schema version.
24
25 ## Toolkit versions
26
docs/scripts/check-toolkit-parity.ts new
+269
@@ -0,0 +1,269 @@
1 +/**
2 + * Checks that the deployed toolkit Markdown pages list the tools production
3 + * serves, for the toolkits with the most tools.
4 + *
5 + * It also checks the toolkit list behind /api/toolkits/instant.
6 + *
7 + * Failures caught:
8 + * - A deployed page lists a tool count matching neither production nor the
9 + * committed snapshot. This is how a stale Next.js data-cache entry shows up:
10 + * tableau.md listed 10 tools for days after production and the snapshot
11 + * both had 436.
12 + * - A production list page at TOOL_PAGE_SIZE would make a cache entry over
13 + * PAGE_BYTES_BUDGET. Next.js does not cache an entry over 2 MiB, and an entry
14 + * holds the body base64-encoded, so a page near 1.5 MiB will go stale once it
15 + * grows past that.
16 + * - The deployed /api/toolkits/instant slugs differ from production, or a
17 + * production /toolkits page exceeds PAGE_BYTES_BUDGET.
18 + *
19 + * Usage: COMPOSIO_API_KEY=... bun scripts/check-toolkit-parity.ts [--top 30] [--markdown report.md]
20 + */
21 +import { writeFile, readFile } from 'fs/promises';
22 +import { fileURLToPath } from 'url';
23 +import { z } from 'zod';
24 +import {
25 + API_PAGE_LIMIT,
26 + createComposioClient,
27 + listToolkits,
28 + listToolkitTools,
29 + TOOL_PAGE_SIZE,
30 + TOOLKIT_PAGE_SIZE,
31 +} from '../lib/composio-api';
32 +import { instantToolkitSlugs } from '../lib/instant-toolkit-api';
33 +import { instantToolkitSlugsSchema } from '../lib/instant-toolkit-filter';
34 +import { fetchWithRetry } from './fetch-with-retry';
35 +
36 +/** The Next.js data-cache entry limit, measured on the serialized entry. */
37 +export const CACHE_ENTRY_LIMIT = 2 * 1024 * 1024;
38 +
39 +/** Alert when an entry reaches 85% of the limit. */
40 +export const PAGE_BYTES_BUDGET = Math.floor(CACHE_ENTRY_LIMIT * 0.85);
41 +
42 +/**
43 + * The size Next.js counts for a cached response: the body is stored
44 + * base64-encoded, so an entry is about 4/3 of the raw body.
45 + */
46 +export function cacheEntryBytes(responseBytes: number): number {
47 + return Math.ceil(responseBytes / 3) * 4;
48 +}
49 +
50 +const DOCS_BASE_URL = 'https://docs.composio.dev';
51 +const SNAPSHOT_PATH = fileURLToPath(new URL('../public/data/toolkits.json', import.meta.url));
52 +
53 +export interface ToolkitObservation {
54 + slug: string;
55 + snapshotCount: number;
56 + productionCount: number;
57 + deployedCount: number;
58 + largestPageBytes: number;
59 +}
60 +
61 +/** Counts the tools listed in the "## Tools" section of a toolkit Markdown page. */
62 +export function countMarkdownTools(markdown: string): number {
63 + const start = markdown.indexOf('\n## Tools\n');
64 + if (start === -1) return 0;
65 + // Tool descriptions can contain headings. The renderer puts triggers after tools.
66 + const triggers = markdown.indexOf('\n## Triggers\n', start + 1);
67 + const section = markdown.slice(start, triggers === -1 ? undefined : triggers);
68 + return section.match(/^\*\*Slug:\*\* `/gm)?.length ?? 0;
69 +}
70 +
71 +/** Describes every way one toolkit's observation breaks parity or the page budget. */
72 +export function findProblems(observation: ToolkitObservation): string[] {
73 + const { slug, snapshotCount, productionCount, deployedCount, largestPageBytes } = observation;
74 + const problems: string[] = [];
75 + if (deployedCount !== productionCount && deployedCount !== snapshotCount) {
76 + problems.push(
77 + `${slug}.md lists ${deployedCount} tools; production has ${productionCount}, the snapshot has ${snapshotCount}`
78 + );
79 + }
80 + if (snapshotCount === API_PAGE_LIMIT && productionCount > API_PAGE_LIMIT) {
81 + problems.push(
82 + `the ${slug} snapshot stops at ${snapshotCount} of ${productionCount} tools, the API page cap; the committed snapshot predates the paginated generator, so re-run generate:toolkits`
83 + );
84 + }
85 + if (cacheEntryBytes(largestPageBytes) > PAGE_BYTES_BUDGET) {
86 + problems.push(
87 + `${slug} has a ${largestPageBytes}-byte tools page (${cacheEntryBytes(largestPageBytes)}-byte cache entry) at TOOL_PAGE_SIZE ${TOOL_PAGE_SIZE}, over the ${PAGE_BYTES_BUDGET}-byte entry budget; lower TOOL_PAGE_SIZE in lib/composio-api.ts`
88 + );
89 + }
90 + return problems;
91 +}
92 +
93 +export interface InstantObservation {
94 + productionSlugs: string[];
95 + deployedSlugs: string[];
96 + largestPageBytes: number;
97 +}
98 +
99 +/** Describes how the deployed Instant toolkit list differs from production. */
100 +export function findInstantProblems({ productionSlugs, deployedSlugs, largestPageBytes }: InstantObservation): string[] {
101 + const problems: string[] = [];
102 + const deployed = new Set(deployedSlugs);
103 + const production = new Set(productionSlugs);
104 + const missing = productionSlugs.filter(slug => !deployed.has(slug));
105 + const extra = deployedSlugs.filter(slug => !production.has(slug));
106 + if (missing.length > 0 || extra.length > 0) {
107 + problems.push(
108 + `/api/toolkits/instant lists ${deployedSlugs.length} toolkits; production has ${productionSlugs.length} (missing: ${missing.slice(0, 5).join(', ') || 'none'}; extra: ${extra.slice(0, 5).join(', ') || 'none'})`
109 + );
110 + }
111 + if (cacheEntryBytes(largestPageBytes) > PAGE_BYTES_BUDGET) {
112 + problems.push(
113 + `a ${largestPageBytes}-byte /toolkits page (${cacheEntryBytes(largestPageBytes)}-byte cache entry) at TOOLKIT_PAGE_SIZE ${TOOLKIT_PAGE_SIZE}, over the ${PAGE_BYTES_BUDGET}-byte entry budget; lower TOOLKIT_PAGE_SIZE in lib/composio-api.ts`
114 + );
115 + }
116 + return problems;
117 +}
118 +
119 +const snapshotSchema = z.array(z.object({ slug: z.string(), toolCount: z.number() }));
120 +
121 +type CountedToolkit = z.infer<typeof snapshotSchema>[number];
122 +
123 +const productionToolkitsSchema = z.array(
124 + z.object({ slug: z.string(), meta: z.object({ tools_count: z.number() }) })
125 +);
126 +
127 +/** Production tool counts from a /toolkits listing, keyed by lowercase slug. */
128 +export function productionToolCounts(items: unknown[]): CountedToolkit[] {
129 + return productionToolkitsSchema
130 + .parse(items)
131 + .map(toolkit => ({ slug: toolkit.slug.toLowerCase(), toolCount: toolkit.meta.tools_count }));
132 +}
133 +
134 +/**
135 + * The toolkits to check: the `top` largest by production count, then the `top`
136 + * largest by snapshot count, without duplicates. Ranking by production too
137 + * keeps a toolkit whose snapshot count collapsed in the set. Only published
138 + * toolkits (those in the snapshot) are eligible, since others have no page.
139 + */
140 +export function selectWatchList(
141 + snapshot: CountedToolkit[],
142 + production: CountedToolkit[],
143 + top: number
144 +): CountedToolkit[] {
145 + const published = new Map(snapshot.map(toolkit => [toolkit.slug, toolkit]));
146 + const largest = (toolkits: CountedToolkit[]) =>
147 + [...toolkits].sort((a, b) => b.toolCount - a.toolCount).slice(0, top);
148 + const fromProduction = largest(production.filter(toolkit => published.has(toolkit.slug))).flatMap(
149 + toolkit => published.get(toolkit.slug) ?? []
150 + );
151 + const watched = new Map<string, CountedToolkit>();
152 + for (const toolkit of [...fromProduction, ...largest(snapshot)]) {
153 + if (!watched.has(toolkit.slug)) watched.set(toolkit.slug, toolkit);
154 + }
155 + return [...watched.values()];
156 +}
157 +
158 +/** A production client that records the size of the largest response body it reads. */
159 +function measuringClient(apiKey: string) {
160 + let largestPageBytes = 0;
161 + const client = createComposioClient(apiKey, {
162 + maxRetries: 0,
163 + timeout: 10 * 60_000,
164 + fetch: async (input, init) => {
165 + const response = await fetchWithRetry(String(input), init);
166 + largestPageBytes = Math.max(largestPageBytes, Buffer.byteLength(await response.clone().text()));
167 + return response;
168 + },
169 + });
170 + return { client, largestPageBytes: () => largestPageBytes };
171 +}
172 +
173 +async function observeProduction(slug: string, apiKey: string) {
174 + const { client, largestPageBytes } = measuringClient(apiKey);
175 + const tools = await listToolkitTools(client, slug);
176 + return { productionCount: tools.length, largestPageBytes: largestPageBytes() };
177 +}
178 +
179 +interface CatalogObservation {
180 + toolkits: unknown[];
181 + largestPageBytes: number;
182 +}
183 +
184 +async function observeCatalog(apiKey: string): Promise<CatalogObservation> {
185 + const { client, largestPageBytes } = measuringClient(apiKey);
186 + const toolkits = await listToolkits(client);
187 + return { toolkits, largestPageBytes: largestPageBytes() };
188 +}
189 +
190 +async function observeInstant({ toolkits, largestPageBytes }: CatalogObservation): Promise<InstantObservation> {
191 + const response = await fetchWithRetry(`${DOCS_BASE_URL}/api/toolkits/instant`);
192 + if (!response.ok) throw new Error(`/api/toolkits/instant responded ${response.status}`);
193 + const deployed = instantToolkitSlugsSchema.parse(await response.json());
194 + return { productionSlugs: instantToolkitSlugs(toolkits), deployedSlugs: deployed.slugs, largestPageBytes };
195 +}
196 +
197 +async function observeDeployed(slug: string): Promise<number> {
198 + const response = await fetchWithRetry(`${DOCS_BASE_URL}/toolkits/${slug}.md`);
199 + if (!response.ok) throw new Error(`${slug}.md responded ${response.status}`);
200 + return countMarkdownTools(await response.text());
201 +}
202 +
203 +function argValue(name: string): string | undefined {
204 + const index = process.argv.indexOf(name);
205 + return index === -1 ? undefined : process.argv[index + 1];
206 +}
207 +
208 +async function main() {
209 + const apiKey = process.env.COMPOSIO_API_KEY;
210 + if (!apiKey) throw new Error('COMPOSIO_API_KEY is required');
211 + const top = Number(argValue('--top') ?? 30);
212 + const markdownPath = argValue('--markdown');
213 +
214 + const snapshot = snapshotSchema.parse(JSON.parse(await readFile(SNAPSHOT_PATH, 'utf-8')));
215 +
216 + const problems: string[] = [];
217 + const rows: string[] = [];
218 +
219 + // One production catalog read feeds both the watch list and the Instant check.
220 + let catalog: CatalogObservation | undefined;
221 + let production: CountedToolkit[] = [];
222 + try {
223 + catalog = await observeCatalog(apiKey);
224 + production = productionToolCounts(catalog.toolkits);
225 + } catch (error) {
226 + problems.push(`Production toolkit catalog: ${error instanceof Error ? error.message : String(error)}`);
227 + }
228 + const toolkits = selectWatchList(snapshot, production, top);
229 + for (const toolkit of toolkits) {
230 + try {
231 + const production = await observeProduction(toolkit.slug, apiKey);
232 + const deployedCount = await observeDeployed(toolkit.slug);
233 + const observation = { slug: toolkit.slug, snapshotCount: toolkit.toolCount, deployedCount, ...production };
234 + problems.push(...findProblems(observation));
235 + rows.push(
236 + `| ${toolkit.slug} | ${toolkit.toolCount} | ${production.productionCount} | ${deployedCount} | ${production.largestPageBytes} |`
237 + );
238 + } catch (error) {
239 + problems.push(`${toolkit.slug}: ${error instanceof Error ? error.message : String(error)}`);
240 + }
241 + }
242 +
243 + if (catalog) {
244 + try {
245 + const instant = await observeInstant(catalog);
246 + problems.push(...findInstantProblems(instant));
247 + rows.push(
248 + `| (Instant toolkit list) | n/a | ${instant.productionSlugs.length} | ${instant.deployedSlugs.length} | ${instant.largestPageBytes} |`
249 + );
250 + } catch (error) {
251 + problems.push(`Instant toolkit list: ${error instanceof Error ? error.message : String(error)}`);
252 + }
253 + }
254 +
255 + const report = [
256 + problems.length === 0 ? '✅ Toolkit pages match production.' : `❌ ${problems.length} toolkit parity problem(s):`,
257 + '',
258 + ...problems.map(problem => `- ${problem}`),
259 + '',
260 + '| Toolkit | Snapshot | Production | Deployed .md | Largest page (bytes) |',
261 + '|---|---|---|---|---|',
262 + ...rows,
263 + ].join('\n');
264 + console.log(report);
265 + if (markdownPath) await writeFile(markdownPath, report);
266 + process.exit(problems.length === 0 ? 0 : 1);
267 +}
268 +
269 +if (import.meta.main) await main();
docs/scripts/fetch-with-retry.ts
+1 -3
@@ -15,9 +15,7 @@
15 *
16 * Why this exists: `generate-toolkits.ts` issues ~6500 requests per run (3 per
17 * toolkit across a ~2.1k catalog), which can exceed the backend request limit.
18 - * Without backoff the run
19 - * fails with `429`, and `generate-meta-tools.ts` (which runs next) inherits the
20 - * exhausted window. See docs/scripts/README.md.
18 + * Without backoff the run can fail with `429`. See docs/scripts/README.md.
19 *
20 * Deliberately hand-rolled rather than using Effect's `Schedule` (which would be
21 * the idiomatic fit elsewhere): the docs package has no Effect dependency and
docs/scripts/generate-meta-tools.ts
+41 -168
@@ -1,161 +1,38 @@
1 /**
2 * Meta Tools Generator Script
3 *
4 - * Fetches meta tool definitions from the Composio Tool Router API and generates:
5 - * - /public/data/meta-tools.json (complete meta tool schemas from API)
4 + * Reads public/data/meta-tools.json and generates:
5 * - /content/toolkits/meta-tools/meta.json (sidebar navigation)
6 * - /content/toolkits/meta-tools/index.mdx (overview page)
7 * - /content/toolkits/meta-tools/{name}.mdx (individual tool pages)
8 *
10 - * All output is derived from the API response — no hardcoded tool definitions.
11 - *
9 * Run: bun run generate:meta-tools
10 *
14 - * Environment variables:
15 - * - COMPOSIO_API_KEY (required)
16 - * - COMPOSIO_API_BASE (optional, must resolve to https://backend.composio.dev/api/v3)
11 + * Set COMPOSIO_META_TOOLS_SCHEMA_URL to refresh the snapshot from a
12 + * production Platform schema export before rendering.
13 */
14
19 -import { writeFile, mkdir, readdir, unlink } from 'fs/promises';
15 +import { writeFile, readFile, mkdir, readdir, unlink } from 'fs/promises';
16 import { join } from 'path';
21 -import { fetchWithRetry } from './fetch-with-retry';
17 +import { metaToolsSchema, type MetaTool } from '../lib/meta-tools-data';
18 import { META_TOOL_OVERRIDES } from '../lib/meta-tool-overrides';
19 import { encodeMarkdownTableCell, encodeYamlString } from '../lib/markdown-escaping';
24 -import { requireProductionApiV3Url, stripStagingHosts } from './production-api.mjs';
25 -import { z } from 'zod';
26 -
27 -const API_BASE = requireProductionApiV3Url(process.env.COMPOSIO_API_BASE);
28 -const API_KEY = process.env.COMPOSIO_API_KEY;
20 +import { STAGING_HOSTS, stripStagingHosts } from './production-api.mjs';
21
22 const DATA_DIR = join(process.cwd(), 'public/data');
23 +const SNAPSHOT_PATH = join(DATA_DIR, 'meta-tools.json');
24 const CONTENT_DIR = join(process.cwd(), 'content/toolkits/meta-tools');
25
33 -interface GeneratedMetaTool {
34 - slug: string;
35 - name: string;
36 - displayName: string;
37 - description: string;
38 - tags: string[];
39 - toolkit: string | null;
40 - inputParameters: Record<string, unknown>;
41 - responseSchema: Record<string, unknown>;
42 -}
43 -
44 -// Zod schemas for the Tool Router payloads this script consumes. Parsing is
45 -// lenient: malformed fields degrade to empty fallbacks instead of aborting.
46 -
47 -const stringArraySchema = z
48 - .array(z.unknown())
49 - .catch([])
50 - .transform(items =>
51 - items.flatMap(item => {
52 - const parsed = z.string().safeParse(item);
53 - return parsed.success ? [parsed.data] : [];
54 - })
55 - );
56 -
57 -const recordSchema = z.record(z.string(), z.unknown()).catch({});
58 -
59 -const sessionResponseSchema = z
60 - .object({
61 - session_id: z.string().optional().catch(undefined),
62 - tool_router_tools: stringArraySchema,
63 - })
64 - .catch({ session_id: undefined, tool_router_tools: [] });
65 -
66 -const metaToolListSchema = z.union([
67 - z.object({ items: z.array(z.unknown()) }).transform(data => data.items),
68 - z.array(z.unknown()),
69 -]);
70 -
71 -const rawMetaToolSchema = z.object({
72 - slug: z.string().catch(''),
73 - name: z.string().catch(''),
74 - description: z.string().catch(''),
75 - tags: stringArraySchema,
76 - toolkit: z.string().optional().catch(undefined),
77 - input_parameters: recordSchema,
78 - output_parameters: recordSchema,
79 -});
80 -
81 -async function createSession(): Promise<string> {
82 - console.log('Creating session...');
83 -
84 - const response = await fetchWithRetry(`${API_BASE}/tool_router/session`, {
85 - method: 'POST',
86 - headers: {
87 - 'Content-Type': 'application/json',
88 - 'x-api-key': API_KEY!,
89 - },
90 - body: JSON.stringify({
91 - user_id: 'default',
92 - config: {
93 - autoManageConnections: true,
94 - recipesEnabled: true,
95 - enableWaitForConnections: true,
96 - },
97 - // Opt-in meta tools are listed only in sessions that enable them.
98 - proxy_execute: { enable: true },
99 - }),
100 - });
101 -
102 - if (!response.ok) {
103 - const body = await response.text();
104 - throw new Error(`Failed to create session: ${response.status} ${response.statusText}\n${body}`);
105 - }
106 -
107 - const data = sessionResponseSchema.parse(await response.json());
108 - const sessionId = data.session_id;
109 -
110 - if (!sessionId) {
111 - throw new Error('No session_id in response');
112 - }
113 -
114 - console.log(` Session created: ${sessionId}`);
115 - console.log(` Tools available: ${data.tool_router_tools.join(', ')}`);
116 - return sessionId;
117 -}
118 -
119 -async function fetchMetaTools(sessionId: string): Promise<unknown[]> {
120 - console.log('Fetching meta tools with schemas...');
121 -
122 - const response = await fetchWithRetry(`${API_BASE}/tool_router/session/${sessionId}/tools`, {
123 - headers: {
124 - 'x-api-key': API_KEY!,
125 - },
126 - });
127 -
128 - if (!response.ok) {
129 - const body = await response.text();
130 - throw new Error(
131 - `Failed to fetch meta tools: ${response.status} ${response.statusText}\n${body}`
132 - );
133 - }
26 +type MetaToolPage = Pick<MetaTool, 'slug' | 'displayName' | 'description'>;
27
135 - const parsed = metaToolListSchema.safeParse(await response.json());
136 - if (!parsed.success) {
137 - throw new Error('Expected array of tools in response');
28 +async function readSchemaExport(source: string): Promise<string> {
29 + const url = new URL(source);
30 + if (url.protocol !== 'https:' || STAGING_HOSTS.includes(url.hostname)) {
31 + throw new Error('COMPOSIO_META_TOOLS_SCHEMA_URL must point to a production HTTPS export');
32 }
139 -
140 - console.log(` Found ${parsed.data.length} meta tools`);
141 - return parsed.data;
142 -}
143 -
144 -export function transformTool(value: unknown): GeneratedMetaTool {
145 - const parsed = rawMetaToolSchema.safeParse(value);
146 - const raw = parsed.success ? parsed.data : rawMetaToolSchema.parse({});
147 - const { slug, name } = raw;
148 -
149 - return {
150 - slug,
151 - name,
152 - displayName: name.replace(/_/g, ' ').replace(/\b\w/g, (c: string) => c.toUpperCase()) || slug,
153 - description: raw.description,
154 - tags: raw.tags,
155 - toolkit: raw.toolkit || null,
156 - inputParameters: raw.input_parameters,
157 - responseSchema: raw.output_parameters,
158 - };
33 + const response = await fetch(url, { redirect: 'error', signal: AbortSignal.timeout(30_000) });
34 + if (!response.ok) throw new Error(`Meta-tool schema export responded ${response.status}`);
35 + return stripStagingHosts(await response.text());
36 }
37
38 /** Derive a short page slug from the tool slug: COMPOSIO_SEARCH_TOOLS -> search_tools */
@@ -175,7 +52,7 @@ export function briefDescription(description: string): string {
52 }
53
54 /** One-line summary for the index table. Prefer hand-written override copy, fall back to the API description. */
178 -function indexLine(tool: GeneratedMetaTool): string {
55 +function indexLine(tool: MetaToolPage): string {
56 const override = META_TOOL_OVERRIDES[tool.slug];
57 if (override) {
58 // First sentence of the hand-written summary keeps the table tight.
@@ -186,7 +63,7 @@ function indexLine(tool: GeneratedMetaTool): string {
63 }
64
65 /** Generate the index.mdx overview page — Modal-voice intro plus a one-line-per-tool table */
189 -export function generateIndexMdx(tools: GeneratedMetaTool[]): string {
66 +export function generateIndexMdx(tools: MetaToolPage[]): string {
67 let content = `---
68 title: Meta Tools
69 description: The system tools every Composio session gives your agent to discover, authenticate, execute, and process tools at runtime.
@@ -220,8 +97,17 @@ These schemas are for reference only. We do not guarantee backward compatibility
97 return content;
98 }
99
100 +/**
101 + * A frontmatter title: plain when it is unambiguous YAML text, quoted otherwise,
102 + * so a remote displayName cannot end the frontmatter or change its meaning.
103 + */
104 +function yamlTitle(value: string): string {
105 + const plain = /^\w(?:[\w ()-]*[\w)])?$/.test(value) && !/^(?:true|false|null|yes|no|on|off|[\d.+-]+)$/i.test(value);
106 + return plain ? value : encodeYamlString(value);
107 +}
108 +
109 /** Generate an individual tool MDX page */
224 -export function generateToolMdx(tool: GeneratedMetaTool): string {
110 +export function generateToolMdx(tool: MetaToolPage): string {
111 // The API's search-tool description contains an outdated catalog count.
112 const description = tool.slug === 'COMPOSIO_SEARCH_TOOLS'
113 ? META_TOOL_OVERRIDES[tool.slug].summary
@@ -229,7 +115,7 @@ export function generateToolMdx(tool: GeneratedMetaTool): string {
115 const desc = encodeYamlString(briefDescription(description));
116
117 return `---
232 -title: ${tool.displayName}
118 +title: ${yamlTitle(tool.displayName)}
119 description: ${desc}
120 keywords: [${tool.slug}, meta tool]
121 ---
@@ -243,7 +129,7 @@ import { MetaToolDetailServer } from '@/components/meta-tools/meta-tool-page';
129 }
130
131 /** Generate meta.json for sidebar navigation — index.mdx is the folder page */
246 -function generateMetaJson(tools: GeneratedMetaTool[]): string {
132 +function generateMetaJson(tools: MetaToolPage[]): string {
133 const pages = tools.map(t => pageSlug(t.slug));
134 return JSON.stringify({ title: 'Meta Tools', defaultOpen: true, pages }, null, 2) + '\n';
135 }
@@ -264,37 +150,29 @@ async function cleanGeneratedMdx() {
150 async function main() {
151 console.log('Starting meta tools generation...\n');
152
267 - await mkdir(DATA_DIR, { recursive: true });
268 - await mkdir(CONTENT_DIR, { recursive: true });
269 -
270 - const sessionId = await createSession();
271 - const rawTools = await fetchMetaTools(sessionId);
272 - const metaTools = rawTools.map(transformTool);
273 -
274 - // Sort alphabetically by slug
153 + const source = process.env.COMPOSIO_META_TOOLS_SCHEMA_URL?.trim();
154 + const contents = source ? await readSchemaExport(source) : await readFile(SNAPSHOT_PATH, 'utf8');
155 + const snapshot: unknown = JSON.parse(contents);
156 + const metaTools = metaToolsSchema.parse(snapshot);
157 metaTools.sort((a, b) => a.slug.localeCompare(b.slug));
158
277 - // 1. Write JSON data
278 - await writeFile(
279 - join(DATA_DIR, 'meta-tools.json'),
280 - stripStagingHosts(JSON.stringify(metaTools, null, 2))
281 - );
282 - console.log(
283 - `\nWrote public/data/meta-tools.json (~${Math.round(JSON.stringify(metaTools).length / 1024)}KB)`
284 - );
159 + if (source) {
160 + await mkdir(DATA_DIR, { recursive: true });
161 + await writeFile(SNAPSHOT_PATH, JSON.stringify(snapshot, null, 2));
162 + console.log('Updated public/data/meta-tools.json from the schema export');
163 + } else {
164 + console.log('Using the checked-in public/data/meta-tools.json snapshot');
165 + }
166
286 - // 2. Clean old generated MDX files and regenerate
167 + await mkdir(CONTENT_DIR, { recursive: true });
168 await cleanGeneratedMdx();
169
289 - // 3. Write meta.json
170 await writeFile(join(CONTENT_DIR, 'meta.json'), generateMetaJson(metaTools));
171 console.log('Wrote content/toolkits/meta-tools/meta.json');
172
293 - // 4. Write index.mdx
173 await writeFile(join(CONTENT_DIR, 'index.mdx'), generateIndexMdx(metaTools));
174 console.log('Wrote content/toolkits/meta-tools/index.mdx');
175
297 - // 5. Write individual tool pages
176 for (const tool of metaTools) {
177 const filename = `${pageSlug(tool.slug)}.mdx`;
178 await writeFile(join(CONTENT_DIR, filename), generateToolMdx(tool));
@@ -306,11 +184,6 @@ async function main() {
184 }
185
186 if (import.meta.main) {
309 - if (!API_KEY) {
310 - console.error('Error: COMPOSIO_API_KEY environment variable is required');
311 - process.exit(1);
312 - }
313 -
187 main().catch(error => {
188 console.error('Fatal error:', error);
189 process.exit(1);
docs/scripts/generate-toolkits.ts
+51 -101
@@ -11,13 +11,22 @@
11 import { mkdir, writeFile } from 'fs/promises';
12 import { join } from 'path';
13 import { fetchWithRetry } from './fetch-with-retry';
14 +import type { Composio } from '@composio/client';
15 +import {
16 + API_PAGE_LIMIT,
17 + createComposioClient,
18 + listToolkits,
19 + listToolkitTools,
20 + listToolkitTriggers,
21 +} from '../lib/composio-api';
22 import { requireProductionApiV3Url, stripStagingHosts } from './production-api.mjs';
23 import { applyToolkitVersions, fetchProductionToolkitVersions } from './toolkit-versions';
24 import { isPublicToolkitSlug } from '../lib/public-toolkit-policy';
25 import { z } from 'zod';
26 import { instantSchema, type Instant } from '../lib/instant';
27
20 -const API_BASE = requireProductionApiV3Url(process.env.COMPOSIO_API_BASE);
28 +// Rejects a non-production override before any request; the client always targets production.
29 +requireProductionApiV3Url(process.env.COMPOSIO_API_BASE);
30 const API_KEY = process.env.COMPOSIO_API_KEY;
31
32 const OUTPUT_DIR = join(process.cwd(), 'public/data');
@@ -120,15 +129,6 @@ const categoryEntrySchema = z.union([
129 z.object({ name: z.string() }).transform(entry => entry.name),
130 ]);
131
123 -// A paged toolkits response: { items, next_cursor } envelope or a bare array.
124 -const toolkitsPageSchema = z.union([
125 - z.object({
126 - items: z.array(z.unknown()),
127 - next_cursor: z.string().optional().catch(undefined),
128 - }),
129 - z.array(z.unknown()).transform(items => ({ items, next_cursor: undefined })),
130 -]);
131 -
132 const slugItemSchema = z.object({ slug: z.string() });
133
134 // Tools/triggers list entries share the slug/name/description shape.
@@ -195,97 +195,65 @@ const authConfigDetailsResponseSchema = z
195 .object({ auth_config_details: z.array(z.unknown()).catch([]) })
196 .catch({ auth_config_details: [] });
197
198 -// The backend silently caps `limit` at 1000 per page and defaults to usage
199 -// ordering, so a single request returns only the top-1000 toolkits — about half
200 -// the catalog. Request the cap and follow `next_cursor` until exhausted.
201 -// MAX_PAGES is a runaway guard well above the real catalog (~2.1k → 3 pages).
202 -const TOOLKITS_PAGE_LIMIT = 1000;
203 -const TOOLKITS_MAX_PAGES = 12;
204 -
205 -export function parseToolkitsPage(value: unknown) {
206 - return toolkitsPageSchema.parse(value);
207 -}
208 -
198 export function parseNamedItems(value: unknown) {
199 return namedItemListSchema.parse(value);
200 }
201
202 +// The backend caps `limit` per page (1000 for toolkits and tools, 50 for
203 +// triggers) and defaults to usage ordering, so a single request returns only
204 +// the first page. Every list request follows `next_cursor` until exhausted.
205 +let client: Composio | undefined;
206 +
207 +/** The production client; fetchWithRetry owns retries, so the client makes none. */
208 +function api(): Composio {
209 + client ??= createComposioClient(API_KEY!, {
210 + maxRetries: 0,
211 + timeout: 10 * 60_000,
212 + fetch: (input, init) => fetchWithRetry(String(input), init),
213 + });
214 + return client;
215 +}
216 +
217 async function fetchToolkits(): Promise<unknown[]> {
218 console.log('Fetching toolkits from API...');
219
216 - const items: unknown[] = [];
220 // Pages can overlap when the catalog shifts between cursor fetches; keep the
221 // first occurrence so API-provided ordering stays stable.
222 const seen = new Set<string>();
220 - let cursor: string | undefined;
221 -
222 - for (let page = 0; page < TOOLKITS_MAX_PAGES; page++) {
223 - const params = new URLSearchParams({ limit: String(TOOLKITS_PAGE_LIMIT) });
224 - if (cursor) params.set('cursor', cursor);
225 -
226 - const response = await fetchWithRetry(`${API_BASE}/toolkits?${params}`, {
227 - headers: {
228 - 'Content-Type': 'application/json',
229 - 'x-api-key': API_KEY!,
230 - },
231 - });
232 -
233 - if (!response.ok) {
234 - throw new Error(`Failed to fetch toolkits: ${response.status} ${response.statusText}`);
235 - }
236 -
237 - const pageData = parseToolkitsPage(await response.json());
238 - for (const item of pageData.items) {
239 - const parsedSlug = slugItemSchema.safeParse(item);
240 - const slug = parsedSlug.success ? parsedSlug.data.slug.toLowerCase() : undefined;
241 - if (slug) {
242 - if (seen.has(slug)) continue;
243 - seen.add(slug);
244 - }
245 - items.push(item);
246 - }
247 -
248 - cursor = pageData.next_cursor;
249 - if (!cursor) return items;
250 - }
251 -
252 - // Failing beats silently publishing a truncated catalog — that is the exact
253 - // bug this pagination loop exists to prevent.
254 - throw new Error(
255 - `Toolkit catalog exceeds ${TOOLKITS_MAX_PAGES} pages of ${TOOLKITS_PAGE_LIMIT}; raise TOOLKITS_MAX_PAGES`
256 - );
223 + return (await listToolkits(api(), { pageSize: API_PAGE_LIMIT })).filter(item => {
224 + const parsedSlug = slugItemSchema.safeParse(item);
225 + if (!parsedSlug.success) return true;
226 + const slug = parsedSlug.data.slug.toLowerCase();
227 + if (seen.has(slug)) return false;
228 + seen.add(slug);
229 + return true;
230 + });
231 }
232
233 +// A failed traversal fails the run: publishing an empty list would replace the
234 +// toolkit's tools in the committed snapshot. Malformed items are still dropped.
235 async function fetchToolsForToolkit(slug: string): Promise<Tool[]> {
260 - const response = await fetchWithRetry(
261 - `${API_BASE}/tools?toolkit_slug=${slug}&toolkit_versions=latest&include_pricing=true&limit=1000`,
262 - {
263 - headers: {
264 - 'Content-Type': 'application/json',
265 - 'x-api-key': API_KEY!,
266 - },
267 - }
268 - );
269 -
270 - if (!response.ok) return [];
271 -
272 - return parseNamedItems(await response.json());
236 + return parseNamedItems(await listToolkitTools(api(), slug, { pageSize: API_PAGE_LIMIT }));
237 }
238
239 async function fetchTriggersForToolkit(slug: string): Promise<Trigger[]> {
276 - const response = await fetchWithRetry(
277 - `${API_BASE}/triggers_types?toolkit_slugs=${slug}&toolkit_versions=latest&limit=1000`,
278 - {
279 - headers: {
280 - 'Content-Type': 'application/json',
281 - 'x-api-key': API_KEY!,
282 - },
283 - }
284 - );
240 + return parseNamedItems(await listToolkitTriggers(api(), slug));
241 +}
242
286 - if (!response.ok) return [];
243 +async function fetchAuthConfigDetails(slug: string): Promise<AuthConfigDetail[]> {
244 + let response: unknown;
245 + try {
246 + response = await api().toolkits.retrieve(slug);
247 + } catch {
248 + return [];
249 + }
250
288 - return parseNamedItems(await response.json());
251 + const data = authConfigDetailsResponseSchema.parse(response);
252 +
253 + return data.auth_config_details.flatMap(item => {
254 + const detail = transformAuthConfigDetail(item);
255 + return detail ? [detail] : [];
256 + });
257 }
258
259 export function transformAuthConfigField(value: unknown, required: boolean): AuthConfigField {
@@ -334,24 +302,6 @@ export function transformAuthConfigDetail(value: unknown): AuthConfigDetail | nu
302 };
303 }
304
337 -async function fetchAuthConfigDetails(slug: string): Promise<AuthConfigDetail[]> {
338 - const response = await fetchWithRetry(`${API_BASE}/toolkits/${slug}`, {
339 - headers: {
340 - 'Content-Type': 'application/json',
341 - 'x-api-key': API_KEY!,
342 - },
343 - });
344 -
345 - if (!response.ok) return [];
346 -
347 - const data = authConfigDetailsResponseSchema.parse(await response.json());
348 -
349 - return data.auth_config_details.flatMap(item => {
350 - const detail = transformAuthConfigDetail(item);
351 - return detail ? [detail] : [];
352 - });
353 -}
354 -
305 export function transformToolkit(raw: unknown): Toolkit {
306 const parsed = rawToolkitSchema.safeParse(raw);
307 const toolkit = parsed.success ? parsed.data : rawToolkitSchema.parse({});
docs/scripts/production-api.mjs
+3 -3
@@ -1,10 +1,10 @@
1 /**
2 * Single source of truth for the Composio production API URL used by the docs
3 - * generators (fetch-openapi.mjs, generate-toolkits.ts, generate-meta-tools.ts).
3 + * toolkit and OpenAPI generators.
4 *
5 * Published docs data must come from production. The "Docs - Update Data"
6 - * workflow leaves the base URL unset, and the toolkit/meta-tool generators
7 - * reject non-production overrides before making a request. Host rewriting is
6 + * workflow leaves the base URL unset, and toolkit generation rejects
7 + * non-production overrides before making a request. Host rewriting is
8 * retained as defense in depth for staging URLs embedded inside an otherwise
9 * production-sourced payload.
10 *
docs/scripts/toolkit-versions.ts
+26 -19
@@ -1,19 +1,25 @@
1 +import { z } from 'zod';
2 import { fetchWithRetry, type FetchWithRetryOptions } from './fetch-with-retry';
2 -import { PRODUCTION_API_V3_URL } from './production-api.mjs';
3 +import { createComposioClient } from '../lib/composio-api';
4
5 interface ToolkitWithVersion {
6 slug: string;
7 version: string | null;
8 }
9
9 -interface ChangelogEntry {
10 - slug?: string;
11 - versions?: Array<{ version?: string }>;
12 -}
10 +// A malformed entry is dropped rather than failing the run; only a response
11 +// without an items array is rejected.
12 +const changelogEntrySchema = z
13 + .object({
14 + slug: z.string().optional().catch(undefined),
15 + versions: z
16 + .array(z.object({ version: z.string().optional().catch(undefined) }).catch({}))
17 + .optional()
18 + .catch(undefined),
19 + })
20 + .catch({});
21
14 -interface ChangelogResponse {
15 - items?: ChangelogEntry[];
16 -}
22 +const changelogSchema = z.object({ items: z.array(changelogEntrySchema) });
23
24 export type ToolkitVersionFetcher = (
25 url: string,
@@ -25,27 +31,28 @@ export async function fetchProductionToolkitVersions(
31 apiKey: string,
32 fetcher: ToolkitVersionFetcher = fetchWithRetry
33 ): Promise<Map<string, string>> {
28 - const response = await fetcher(`${PRODUCTION_API_V3_URL}/toolkits/changelog`, {
29 - headers: {
30 - 'Content-Type': 'application/json',
31 - 'x-api-key': apiKey,
32 - },
34 + const client = createComposioClient(apiKey, {
35 + maxRetries: 0,
36 + timeout: 10 * 60_000,
37 + fetch: (input, init) => fetcher(String(input), init),
38 });
39
35 - if (!response.ok) {
40 + let response: unknown;
41 + try {
42 + response = await client.toolkits.retrieveChangelog();
43 + } catch (error) {
44 throw new Error(
37 - `Failed to fetch production toolkit changelog: ${response.status} ${response.statusText}`
45 + `Failed to fetch production toolkit changelog: ${error instanceof Error ? error.message : String(error)}`
46 );
47 }
40 -
41 - const data = (await response.json()) as ChangelogResponse;
42 - if (!Array.isArray(data.items)) {
48 + const data = changelogSchema.safeParse(response);
49 + if (!data.success) {
50 throw new Error('Production toolkit changelog response is missing an items array');
51 }
52
53 const versionMap = new Map<string, string>();
54
48 - for (const entry of data.items) {
55 + for (const entry of data.data.items) {
56 const slug = entry.slug?.toLowerCase();
57 const latestVersion = entry.versions?.[0]?.version;
58 if (slug && latestVersion) {
docs/tests/static/check-toolkit-parity.test.ts new
+155
@@ -0,0 +1,155 @@
1 +import { describe, expect, test } from 'bun:test';
2 +import {
3 + countMarkdownTools,
4 + cacheEntryBytes,
5 + CACHE_ENTRY_LIMIT,
6 + findInstantProblems,
7 + findProblems,
8 + PAGE_BYTES_BUDGET,
9 + productionToolCounts,
10 + selectWatchList,
11 +} from '../../scripts/check-toolkit-parity';
12 +
13 +const markdown = [
14 + '# Tableau',
15 + '',
16 + '- **Tools:** 436',
17 + '',
18 + '## Frequently Asked Questions',
19 + '',
20 + '### Does it support OAuth?',
21 + '',
22 + '## Tools',
23 + '',
24 + '### List Users',
25 + '',
26 + '**Slug:** `TABLEAU_LIST_USERS`',
27 + '',
28 + '## Pagination',
29 + '',
30 + 'Use the cursor to retrieve more users.',
31 + '',
32 + '### List Views',
33 + '',
34 + '**Slug:** `TABLEAU_LIST_VIEWS`',
35 + '',
36 + '## Triggers',
37 + '',
38 + '### New View',
39 + '',
40 + '**Slug:** `TABLEAU_NEW_VIEW_TRIGGER`',
41 +].join('\n');
42 +
43 +const healthy = {
44 + slug: 'tableau',
45 + snapshotCount: 436,
46 + productionCount: 436,
47 + deployedCount: 436,
48 + largestPageBytes: 1_231_773,
49 +};
50 +
51 +describe('countMarkdownTools', () => {
52 + test('counts tools after a heading in a description and excludes triggers', () => {
53 + expect(countMarkdownTools(markdown)).toBe(2);
54 + });
55 +
56 + test('returns 0 for a page without a Tools section', () => {
57 + expect(countMarkdownTools('# Empty toolkit\n\n## Triggers\n\n**Slug:** `X`')).toBe(0);
58 + });
59 +});
60 +
61 +describe('findProblems', () => {
62 + test('accepts a page that matches production and the snapshot', () => {
63 + expect(findProblems(healthy)).toEqual([]);
64 + });
65 +
66 + test('flags the stale tableau.md listing 10 of 436 tools', () => {
67 + expect(findProblems({ ...healthy, deployedCount: 10 })).toEqual([
68 + 'tableau.md lists 10 tools; production has 436, the snapshot has 436',
69 + ]);
70 + });
71 +
72 + test('accepts a page that still matches a snapshot awaiting its data-sync PR', () => {
73 + expect(findProblems({ ...healthy, snapshotCount: 10, deployedCount: 10 })).toEqual([]);
74 + });
75 +
76 + test('flags a snapshot cut off at the API page cap', () => {
77 + expect(findProblems({ ...healthy, slug: 'microsoft_365', snapshotCount: 1000, productionCount: 1208, deployedCount: 1208 })).toEqual([
78 + 'the microsoft_365 snapshot stops at 1000 of 1208 tools, the API page cap; the committed snapshot predates the paginated generator, so re-run generate:toolkits',
79 + ]);
80 + });
81 +
82 + test('measures the page budget on the base64 cache entry, not the raw body', () => {
83 + // 1.6 MB raw stays under 2 MiB, but its base64 entry (2.13 MB) does not fit.
84 + expect(cacheEntryBytes(1_600_000)).toBeGreaterThan(CACHE_ENTRY_LIMIT);
85 + expect(findProblems({ ...healthy, largestPageBytes: 1_600_000 })).toHaveLength(1);
86 + expect(cacheEntryBytes(healthy.largestPageBytes)).toBeLessThanOrEqual(PAGE_BYTES_BUDGET);
87 + });
88 +});
89 +
90 +describe('findInstantProblems', () => {
91 + const instant = { productionSlugs: ['exa', 'tavily'], deployedSlugs: ['tavily', 'exa'], largestPageBytes: 400_000 };
92 +
93 + test('accepts the same slugs in any order', () => {
94 + expect(findInstantProblems(instant)).toEqual([]);
95 + });
96 +
97 + test('flags a deployed list frozen before a toolkit gained Instant support', () => {
98 + expect(findInstantProblems({ ...instant, deployedSlugs: ['exa'] })).toEqual([
99 + '/api/toolkits/instant lists 1 toolkits; production has 2 (missing: tavily; extra: none)',
100 + ]);
101 + });
102 +
103 + test('flags a production /toolkits page whose cache entry is over budget', () => {
104 + expect(findInstantProblems({ ...instant, largestPageBytes: 1_600_000 })).toHaveLength(1);
105 + });
106 +});
107 +
108 +describe('selectWatchList', () => {
109 + const snapshot = [
110 + { slug: 'github', toolCount: 905 },
111 + { slug: 'tableau', toolCount: 10 },
112 + { slug: 'gmail', toolCount: 60 },
113 + { slug: 'slack', toolCount: 150 },
114 + ];
115 +
116 + test('keeps a toolkit whose snapshot count collapsed, using production counts', () => {
117 + const production = [
118 + { slug: 'github', toolCount: 905 },
119 + { slug: 'tableau', toolCount: 436 },
120 + { slug: 'gmail', toolCount: 60 },
121 + { slug: 'slack', toolCount: 150 },
122 + ];
123 +
124 + expect(selectWatchList(snapshot, production, 2).map(toolkit => toolkit.slug)).toEqual([
125 + 'github',
126 + 'tableau',
127 + 'slack',
128 + ]);
129 + });
130 +
131 + test('carries snapshot counts and skips production toolkits without a published page', () => {
132 + const production = [{ slug: 'unpublished', toolCount: 5000 }, { slug: 'tableau', toolCount: 436 }];
133 +
134 + expect(selectWatchList(snapshot, production, 1)).toEqual([
135 + { slug: 'tableau', toolCount: 10 },
136 + { slug: 'github', toolCount: 905 },
137 + ]);
138 + });
139 +
140 + test('falls back to the snapshot ranking without production counts', () => {
141 + expect(selectWatchList(snapshot, [], 2).map(toolkit => toolkit.slug)).toEqual(['github', 'slack']);
142 + });
143 +});
144 +
145 +describe('productionToolCounts', () => {
146 + test('reads meta.tools_count and lowercases slugs', () => {
147 + expect(productionToolCounts([{ slug: 'TABLEAU', meta: { tools_count: 436 } }])).toEqual([
148 + { slug: 'tableau', toolCount: 436 },
149 + ]);
150 + });
151 +
152 + test('rejects a listing without tool counts', () => {
153 + expect(() => productionToolCounts([{ slug: 'tableau' }])).toThrow();
154 + });
155 +});
docs/tests/static/composio-api.test.ts new
+50
@@ -0,0 +1,50 @@
1 +import { describe, expect, mock, test } from 'bun:test';
2 +import {
3 + createComposioClient,
4 + listToolkitTools,
5 + listToolkitTriggers,
6 + TOOL_PAGE_SIZE,
7 + TRIGGER_PAGE_LIMIT,
8 +} from '../../lib/composio-api';
9 +
10 +type Init = RequestInit & { next?: { revalidate?: number } };
11 +
12 +function recordingClient(respond: (url: URL) => Response) {
13 + const fetcher = mock(async (input: RequestInfo | URL, _init?: Init) => respond(new URL(String(input))));
14 + const client = createComposioClient('test-key', { revalidate: 3600, maxRetries: 0, fetch: fetcher });
15 + return { client, fetcher };
16 +}
17 +
18 +const page = (items: unknown[], next_cursor: string | null) =>
19 + Response.json({ items, next_cursor, total_pages: 2, current_page: next_cursor ? 1 : 2, total_items: 3 });
20 +
21 +describe('listToolkitTools', () => {
22 + test('follows next_cursor at TOOL_PAGE_SIZE, one cached request per page', async () => {
23 + const { client, fetcher } = recordingClient(url =>
24 + url.searchParams.get('cursor') === 'c2' ? page([{ slug: 'C' }], null) : page([{ slug: 'A' }, { slug: 'B' }], 'c2')
25 + );
26 +
27 + expect(await listToolkitTools(client, 'tableau')).toEqual([{ slug: 'A' }, { slug: 'B' }, { slug: 'C' }]);
28 +
29 + const urls = fetcher.mock.calls.map(([input]) => new URL(String(input)));
30 + expect(urls.map(url => url.pathname)).toEqual(['/api/v3.1/tools', '/api/v3.1/tools']);
31 + expect(urls.map(url => url.searchParams.get('toolkit_slug'))).toEqual(['TABLEAU', 'TABLEAU']);
32 + expect(urls.map(url => url.searchParams.get('limit'))).toEqual([String(TOOL_PAGE_SIZE), String(TOOL_PAGE_SIZE)]);
33 + expect(urls.map(url => url.searchParams.get('cursor'))).toEqual([null, 'c2']);
34 + expect(fetcher.mock.calls.map(([, init]) => new Headers(init?.headers).get('x-api-key'))).toEqual(['test-key', 'test-key']);
35 + expect(fetcher.mock.calls.map(([, init]) => init?.next)).toEqual([{ revalidate: 3600 }, { revalidate: 3600 }]);
36 + });
37 +});
38 +
39 +describe('listToolkitTriggers', () => {
40 + test('requests the latest trigger types of one toolkit', async () => {
41 + const { client, fetcher } = recordingClient(() => page([{ slug: 'GITHUB_PUSH' }], null));
42 +
43 + expect(await listToolkitTriggers(client, 'github')).toEqual([{ slug: 'GITHUB_PUSH' }]);
44 + const url = new URL(String(fetcher.mock.calls[0]?.[0]));
45 + expect(url.pathname).toBe('/api/v3.1/triggers_types');
46 + expect(url.searchParams.get('toolkit_slugs')).toBe('GITHUB');
47 + expect(url.searchParams.get('toolkit_versions')).toBe('latest');
48 + expect(url.searchParams.get('limit')).toBe(String(TRIGGER_PAGE_LIMIT));
49 + });
50 +});
docs/tests/static/generate-toolkits.test.ts
+18 -65
@@ -17,7 +17,6 @@ import { describe, test, expect } from "bun:test";
17 import {
18 authConfigFields,
19 parseNamedItems,
20 - parseToolkitsPage,
20 shouldPublishToolkit,
21 transformAuthConfigField,
22 transformAuthConfigDetail,
@@ -26,7 +25,6 @@ import {
25 import {
26 generateIndexMdx,
27 generateToolMdx,
29 - transformTool,
28 } from "../../scripts/generate-meta-tools";
29
30 describe("transformToolkit", () => {
@@ -296,83 +294,38 @@ describe("authConfigFields", () => {
294 });
295 });
296
299 -// generate-meta-tools.ts's transformTool is a cheap, pure sibling transform
300 -// built on the same lenient zod parsing; a couple of cases here mirror the
301 -// coverage above without pulling in createSession/fetchMetaTools (network).
302 -describe("transformTool (generate-meta-tools.ts)", () => {
303 - test("maps a representative full meta tool object end to end", () => {
304 - const tool = transformTool({
305 - slug: "COMPOSIO_SEARCH_TOOLS",
306 - name: "composio_search_tools",
307 - description: "Searches for tools",
308 - tags: ["discovery", "search", 42],
309 - toolkit: "meta",
310 - input_parameters: { type: "object" },
311 - output_parameters: { type: "object" },
312 - });
313 -
314 - expect(tool.slug).toBe("COMPOSIO_SEARCH_TOOLS");
315 - expect(tool.name).toBe("composio_search_tools");
316 - expect(tool.displayName).toBe("Composio Search Tools");
317 - expect(tool.description).toBe("Searches for tools");
318 - expect(tool.tags).toEqual(["discovery", "search"]);
319 - expect(tool.toolkit).toBe("meta");
320 - expect(tool.inputParameters).toEqual({ type: "object" });
321 - expect(tool.responseSchema).toEqual({ type: "object" });
322 - });
323 -
324 - test("toolkit falls back to null when absent or non-string", () => {
325 - expect(transformTool({ slug: "x", name: "x" }).toolkit).toBeNull();
326 - expect(transformTool({ slug: "x", name: "x", toolkit: "" }).toolkit).toBeNull();
327 - expect(transformTool({ slug: "x", name: "x", toolkit: 7 }).toolkit).toBeNull();
328 - });
329 -
330 - test("displayName falls back to the slug when name is empty", () => {
331 - const tool = transformTool({ slug: "COMPOSIO_X", name: "" });
332 - expect(tool.displayName).toBe("COMPOSIO_X");
333 - });
334 -
335 - test("malformed/empty input objects do not throw", () => {
336 - expect(() => transformTool({})).not.toThrow();
337 - expect(() => transformTool(null)).not.toThrow();
338 - expect(() => transformTool(undefined)).not.toThrow();
339 -
340 - const tool = transformTool({});
341 - expect(tool.slug).toBe("");
342 - expect(tool.name).toBe("");
343 - expect(tool.displayName).toBe("");
344 - expect(tool.tags).toEqual([]);
345 - expect(tool.toolkit).toBeNull();
346 - expect(tool.inputParameters).toEqual({});
347 - expect(tool.responseSchema).toEqual({});
348 - });
349 -
297 +describe("generated meta-tool MDX", () => {
298 test("encodes table delimiters in generated index descriptions", () => {
351 - const tool = transformTool({
299 + const tool = {
300 slug: "COMPOSIO_X",
353 - name: "composio_x",
301 + displayName: "Composio X",
302 description: String.raw`Compare \| and | values`,
355 - });
303 + };
304
305 expect(generateIndexMdx([tool])).toContain(String.raw`Compare \&#124; and &#124; values`);
306 });
307
308 test("serializes generated frontmatter descriptions as YAML strings", () => {
361 - const tool = transformTool({
309 + const tool = {
310 slug: "COMPOSIO_X",
363 - name: "composio_x",
311 + displayName: "Composio X",
312 description: 'Use "quotes" at C:\\tools',
365 - });
313 + };
314
315 expect(generateToolMdx(tool)).toContain('description: "Use \\"quotes\\" at C:\\\\tools"');
316 });
369 -});
317
371 -describe("parseToolkitsPage", () => {
372 - test("rejects malformed page envelopes instead of returning an empty page", () => {
373 - expect(() => parseToolkitsPage({ error: "rate limited" })).toThrow();
374 - expect(() => parseToolkitsPage({ items: null })).toThrow();
375 - expect(() => parseToolkitsPage("not a page")).toThrow();
318 + test("serializes generated frontmatter titles as YAML strings", () => {
319 + const tool = {
320 + slug: "COMPOSIO_X",
321 + displayName: 'Search: "Advanced" Tools',
322 + description: "Searches",
323 + };
324 +
325 + expect(generateToolMdx(tool)).toContain('title: "Search: \\"Advanced\\" Tools"\n');
326 + expect(generateToolMdx({ ...tool, displayName: "Line\n---\n{evil}" })).toContain('title: "Line\\n---\\n{evil}"\n');
327 + expect(generateToolMdx({ ...tool, displayName: "null" })).toContain('title: "null"\n');
328 + expect(generateToolMdx({ ...tool, displayName: "Search Tools" })).toContain("title: Search Tools\n");
329 });
330 });
331
docs/tests/static/instant-toolkit-filter.test.tsx
+5 -3
@@ -3,6 +3,7 @@ import { renderToStaticMarkup } from 'react-dom/server';
3 import { readFileSync } from 'node:fs';
4 import { filterToolkits, withInstantEligibility } from '../../lib/instant-toolkit-filter';
5 import { fetchInstantToolkitSlugs } from '../../lib/instant-toolkit-api';
6 +import { TOOLKIT_PAGE_SIZE } from '../../lib/composio-api';
7 import { InstantBolt } from '../../components/toolkits/instant-bolt';
8 import { InstantBadge } from '../../components/toolkits/instant-badge';
9
@@ -52,14 +53,15 @@ describe('live Instant eligibility', () => {
53 const item = (slug: string, supported = true) => ({ slug, type: 'native', enabled: true, instant: { supported } });
54 const fakeFetch = (fn: (input: Parameters<typeof fetch>[0], init?: RequestInit) => Promise<Response>) => Object.assign(mock(fn), { preconnect: fetch.preconnect });
55
55 - test('follows cursors, requests pricing, deduplicates and only accepts explicit support', async () => {
56 + test('follows cursors, deduplicates and only accepts explicit support', async () => {
57 const fetcher = fakeFetch(async input => String(input).includes('cursor=next')
58 ? Response.json({ items: [item('ALPHA'), item('beta'), item('no', false)], next_cursor: null })
59 : Response.json({ items: [item('alpha'), { slug: 'null', instant: null }, { slug: 'missing' }], next_cursor: 'next' }));
60 expect(await fetchInstantToolkitSlugs('test-key', fetcher)).toEqual(['alpha', 'beta']);
61 expect(fetcher).toHaveBeenCalledTimes(2);
61 - expect(String(fetcher.mock.calls[0][0])).toContain('include_pricing=true');
62 - expect(fetcher.mock.calls[0][1]?.headers).toEqual({ 'x-api-key': 'test-key' });
62 + expect(new URL(String(fetcher.mock.calls[0][0])).pathname).toBe('/api/v3.1/toolkits');
63 + expect(fetcher.mock.calls.map(([input]) => new URL(String(input)).searchParams.get('limit'))).toEqual([String(TOOLKIT_PAGE_SIZE), String(TOOLKIT_PAGE_SIZE)]);
64 + expect(new Headers(fetcher.mock.calls[0][1]?.headers).get('x-api-key')).toBe('test-key');
65 });
66
67 test('partial responses, missing eligibility, malformed data and repeated cursors fail rather than imply no results', async () => {
docs/tests/static/production-versions.test.ts
+18 -1
@@ -85,7 +85,7 @@ describe("Toolkit version source - production only", () => {
85 const versions = await fetchProductionToolkitVersions("test-key", fetcher);
86
87 expect(requests).toEqual([
88 - "https://backend.composio.dev/api/v3/toolkits/changelog",
88 + "https://backend.composio.dev/api/v3.1/toolkits/changelog",
89 ]);
90 expect(versions.get("gmail")).toBe("20260702_01");
91 });
@@ -98,6 +98,23 @@ describe("Toolkit version source - production only", () => {
98 );
99 });
100
101 + test("skips malformed changelog entries instead of failing the run", async () => {
102 + const fetcher: ToolkitVersionFetcher = async () =>
103 + Response.json({
104 + items: [
105 + null,
106 + { slug: 7, versions: [{ version: "20260101_01" }] },
107 + { slug: "SLACK", versions: "none" },
108 + { slug: "NOTION", versions: [{ version: 3 }] },
109 + { slug: "GMAIL", versions: [{ version: "20260702_01" }] },
110 + ],
111 + });
112 +
113 + const versions = await fetchProductionToolkitVersions("test-key", fetcher);
114 +
115 + expect([...versions]).toEqual([["gmail", "20260702_01"]]);
116 + });
117 +
118 test("rejects an empty production version map before rewriting the catalog", async () => {
119 const fetcher: ToolkitVersionFetcher = async () => Response.json({ items: [] });
120
docs/tests/static/proxy-execute-meta-tool.test.tsx
+53 -19
@@ -1,5 +1,5 @@
1 import { describe, expect, test } from 'bun:test';
2 -import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';
2 +import { mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';
3 import { tmpdir } from 'node:os';
4 import { join, resolve } from 'node:path';
5 import { renderToStaticMarkup } from 'react-dom/server';
@@ -9,43 +9,77 @@ import { MetaToolDetail } from '../../components/meta-tools/meta-tool-detail';
9 import { getMetaToolBySlug } from '../../lib/meta-tools-data';
10
11 describe('proxy execute meta-tool reference', () => {
12 - test('generation discovers opt-in tools and preserves unrestricted JSON', async () => {
12 + const exportUrl = 'https://backend.composio.dev/meta-tools.json';
13 + const tool = {
14 + slug: 'COMPOSIO_PROXY_EXECUTE',
15 + name: 'proxy_execute',
16 + displayName: 'Proxy Execute',
17 + description: 'Read from the snapshot',
18 + tags: [],
19 + toolkit: null,
20 + inputParameters: { type: 'object', properties: { body: { description: 'Any JSON' } } },
21 + responseSchema: { type: 'object', properties: {} },
22 + };
23 +
24 + test.each([
25 + ['checked-in snapshot', ''],
26 + ['read-only export', exportUrl],
27 + ['invalid export', `${exportUrl}?empty`],
28 + ])('generation handles the %s', async (_name, source) => {
29 const directory = await mkdtemp(join(tmpdir(), 'composio-meta-tools-'));
30 const transport = join(directory, 'transport.ts');
15 - await writeFile(transport, `
16 - let enabled = false;
17 - globalThis.fetch = async (_url, options) => {
18 - if (options?.method === 'POST') {
19 - enabled = JSON.parse(options.body).proxy_execute?.enable === true;
20 - return Response.json({ session_id: 'test-session', tool_router_tools: [] });
21 - }
22 - return Response.json(enabled ? [{
23 - slug: 'COMPOSIO_PROXY_EXECUTE', name: 'proxy_execute',
24 - input_parameters: { type: 'object', properties: { body: { description: 'Any JSON' } } },
25 - output_parameters: { type: 'object', properties: {} },
26 - }] : []);
27 - };
28 - `);
31 + const snapshotPath = join(directory, 'public/data/meta-tools.json');
32 + const snapshot = JSON.stringify([tool]);
33 + const exportedTool = { ...tool, description: 'Read from the export' };
34 + const invalidExport = source.endsWith('?empty');
35
36 try {
37 + await mkdir(join(directory, 'public/data'), { recursive: true });
38 + await writeFile(snapshotPath, snapshot);
39 + await writeFile(transport, `
40 + globalThis.fetch = async (input, init) => {
41 + const request = new Request(input, init);
42 + const source = process.env.COMPOSIO_META_TOOLS_SCHEMA_URL;
43 + if (!source || request.url !== source || request.method !== 'GET' ||
44 + request.headers.has('x-api-key') || request.headers.has('authorization')) {
45 + throw new Error('Unexpected request from the meta-tool generator');
46 + }
47 + return Response.json(${JSON.stringify(invalidExport ? [] : [exportedTool])});
48 + };
49 + `);
50 const child = Bun.spawn([
51 process.execPath, '--preload', transport, resolve('scripts/generate-meta-tools.ts'),
52 ], {
53 cwd: directory,
35 - env: { ...process.env, COMPOSIO_API_KEY: 'test-key', COMPOSIO_API_BASE: '' },
54 + env: {
55 + ...process.env,
56 + COMPOSIO_API_KEY: source ? 'test-key' : '',
57 + COMPOSIO_META_TOOLS_SCHEMA_URL: source,
58 + },
59 stdout: 'pipe',
60 stderr: 'pipe',
61 });
62 const [exitCode, stdout, stderr] = await Promise.all([
63 child.exited, new Response(child.stdout).text(), new Response(child.stderr).text(),
64 ]);
65 + if (invalidExport) {
66 + expect(exitCode).toBe(1);
67 + expect(stderr).toContain('ZodError');
68 + expect(await readFile(snapshotPath, 'utf8')).toBe(snapshot);
69 + return;
70 + }
71 +
72 expect(exitCode, `${stdout}\n${stderr}`).toBe(0);
73 expect(await readFile(join(directory, 'content/toolkits/meta-tools/index.mdx'), 'utf8'))
74 .toContain('/toolkits/meta-tools/proxy_execute');
75 expect(await readFile(join(directory, 'content/toolkits/meta-tools/proxy_execute.mdx'), 'utf8'))
76 .toContain('<MetaToolDetailServer slug="COMPOSIO_PROXY_EXECUTE" />');
47 - const tools = await Bun.file(join(directory, 'public/data/meta-tools.json')).json();
48 - expect(tools[0].inputParameters.properties.body).toEqual({ description: 'Any JSON' });
77 + const saved = await readFile(snapshotPath, 'utf8');
78 + if (source) {
79 + expect(JSON.parse(saved)).toEqual([exportedTool]);
80 + } else {
81 + expect(saved).toBe(snapshot);
82 + }
83 } finally {
84 await rm(directory, { recursive: true, force: true });
85 }
docs/tests/static/toolkit-api.test.ts
+15 -12
@@ -125,12 +125,11 @@ describe('fetchToolkitFromProduction', () => {
125 ],
126 });
127 expect(fetcher).toHaveBeenCalledTimes(1);
128 - expect(fetcher.mock.calls[0]?.[0]).toBe('https://backend.composio.dev/api/v3/toolkits/github');
129 - expect(fetcher.mock.calls[0]?.[1]).toEqual({
130 - headers: { 'Content-Type': 'application/json', 'x-api-key': 'test-key' },
131 - next: { revalidate: 3600 },
132 - signal: expect.any(AbortSignal),
133 - });
128 + expect(fetcher.mock.calls[0]?.[0]).toBe('https://backend.composio.dev/api/v3.1/toolkits/github');
129 + const init = fetcher.mock.calls[0]?.[1] as (RequestInit & { next?: unknown }) | undefined;
130 + expect(new Headers(init?.headers).get('x-api-key')).toBe('test-key');
131 + expect(init?.next).toEqual({ revalidate: 3600 });
132 + expect(init?.signal).toBeInstanceOf(AbortSignal);
133 });
134
135 test('matches the committed snapshot key set and metadata defaults', async () => {
@@ -293,13 +292,17 @@ describe('fetchToolkitFromProduction', () => {
292 useApiKey();
293 const controller = new AbortController();
294 const timeout = spyOn(AbortSignal, 'timeout').mockReturnValue(controller.signal);
296 - const fetcher = mock(async () => Response.json(productionPayload));
297 - globalThis.fetch = fetcher;
298 -
299 - await fetchToolkitFromProduction('github');
295 + spyOn(console, 'warn').mockImplementation(() => {});
296 + let sent: AbortSignal | null | undefined;
297 + globalThis.fetch = mock(async (_input: RequestInfo | URL, init?: RequestInit) => {
298 + sent = init?.signal;
299 + controller.abort();
300 + return Response.json(productionPayload);
301 + });
302
303 + expect(await fetchToolkitFromProduction('timeout-bound')).toBeNull();
304 expect(timeout).toHaveBeenCalledWith(15_000);
302 - expect(fetcher.mock.calls[0]?.[1]?.signal).toBe(controller.signal);
305 + expect(sent?.aborted).toBe(true);
306 });
307
308 test('requires an API key and warns without fetching', async () => {
@@ -333,7 +336,7 @@ describe('fetchToolkitFromProduction', () => {
336 globalThis.fetch = fetcher;
337
338 await fetchToolkitFromProduction('github');
336 - expect(fetcher.mock.calls[0]?.[0]).toBe('https://backend.composio.dev/api/v3/toolkits/github');
339 + expect(fetcher.mock.calls[0]?.[0]).toBe('https://backend.composio.dev/api/v3.1/toolkits/github');
340 });
341
342 test('negative-caches a 404 for 60 seconds, then retries', async () => {