@devries-cameron20 / composio Mirror · read-only
Fork 0

Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

Mirrored from https://github.com/Camo67/composio.git

Synced Nov 21, 03:37 UTC

next
md 111 lines 3.92 KB
Rendered Raw
1 # OpenAPI Scripts
2
3 ## fetch-with-retry.ts
4
5 Shared `fetchWithRetry` helper used by the toolkit generator and parity checker.
6 It wraps the global `fetch`
7 with rate-limit-aware retry/backoff: it retries `429` and transient 5xx
8 responses, honors the `Retry-After` header when present, otherwise falls back to
9 exponential backoff with jitter, and caps attempts so CI still fails fast when
10 the backend is genuinely down.
11
12 This matters because `generate-toolkits.ts` issues ~6500 requests per run
13 (a few catalog pages + 3 per toolkit across a ~2.1k catalog), which can exceed
14 the backend request limit.
15
16 ## Meta-tools reference
17
18 `bun run generate:meta-tools` renders the reference from
19 `public/data/meta-tools.json`. Set `COMPOSIO_META_TOOLS_SCHEMA_URL` to the
20 production Platform HTTPS export to update the snapshot before rendering. The
21 export must return the same JSON array format as the snapshot. Docs - Update Data
22 reads the URL from the repository variable of the same name. With the variable
23 unset, reference pages use the checked-in schema version.
24
25 ## Toolkit versions
26
27 `generate-toolkits.ts` owns the complete toolkit catalog and always sources it
28 from the production API. For a version-only repair, run the narrower generator:
29
30 ```bash
31 COMPOSIO_API_KEY=... bun run generate:toolkit-versions
32 ```
33
34 Both paths share `toolkit-versions.ts`, so they fetch and apply changelog values
35 with identical semantics: a toolkit missing from the production changelog gets
36 `version: null`. Any `COMPOSIO_API_BASE` override must normalize to
37 `https://backend.composio.dev/api/v3`; non-production sources fail before a
38 request is made.
39
40 ## fetch-openapi.mjs
41
42 Fetches the Composio OpenAPI spec and filters it for use in Fumadocs API reference documentation.
43
44 ### Usage
45
46 ```bash
47 bun run scripts/fetch-openapi.mjs
48 ```
49
50 This outputs `public/openapi.json` which is used by `lib/openapi.ts`.
51
52 ### Why Filtering is Needed
53
54 The raw OpenAPI spec from `https://backend.composio.dev/api/v3/openapi.json` has issues that break documentation generators:
55
56 1. **Endpoints with multiple tags** - Causes duplicate entries in sidebar
57 2. **Internal endpoints exposed** - CLI, Admin, Profiling endpoints shouldn't be in public docs
58
59 See `OPENAPI_IMPROVEMENTS.md` in the fumadocs root for planned fixes to the spec itself.
60
61 ### What Gets Filtered
62
63 #### Ignored Paths
64 These endpoints are completely removed:
65 - `/api/v3/mcp/validate/{uuid}`
66 - `/api/v3/cli/get-session`
67 - `/api/v3/cli/create-session`
68 - `/api/v3/auth/session/logout`
69
70 #### Ignored Tags
71 Endpoints with only these tags are removed:
72 - `CLI`
73 - `Admin`
74 - `Profiling`
75
76 #### Duplicate Prevention
77 If an endpoint has multiple tags, only the first tag is kept. This prevents the same endpoint appearing in multiple sidebar sections.
78
79 ### Configuration
80
81 The script uses environment variables:
82
83 | Variable | Default | Description |
84 |----------|---------|-------------|
85 | `OPENAPI_SPEC_URL` | `https://backend.composio.dev/api/v3/openapi.json` | Source OpenAPI spec URL |
86
87 For staging deployments, set:
88 ```bash
89 OPENAPI_SPEC_URL=https://staging.composio.dev/api/v3/openapi.json
90 ```
91
92 ## verify-kb.ts
93
94 Checks published KB guides against the production data this repo already
95 refreshes. Runs inside `docs-update-data.yml` right after the catalog and
96 OpenAPI specs are regenerated, so freshness is verified on every production
97 deploy instead of on a review calendar.
98
99 ```bash
100 bun run verify:kb # catalog checks only, no network
101 bun run verify:kb --check-links # also probe provider doc links
102 bun run verify:kb --json # machine-readable report
103 bun run verify:kb --markdown out.md # write the report to a file
104 ```
105
106 Exits non-zero when any finding is an error. `--warnings-as-errors` promotes
107 warnings too. See `docs/decisions/kb-freshness-verification.md` for what is
108 checked and why placeholders, bare URLs, and 5xx responses are excluded.
109
110 A guide that deliberately cites a removed identifier declares it in
111 `verifyIgnoreToolSlugs` in its `kb/guides/<slug>.json`.