Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.
Mirrored from https://github.com/Camo67/composio.git
Synced Nov 21, 03:37 UTC
| 1 | # OpenAPI Scripts |
| 2 | |
| 3 | ## fetch-with-retry.ts |
| 4 | |
| 5 | Shared `fetchWithRetry` helper used by the toolkit generator and parity checker. |
| 6 | It wraps the global `fetch` |
| 7 | with rate-limit-aware retry/backoff: it retries `429` and transient 5xx |
| 8 | responses, honors the `Retry-After` header when present, otherwise falls back to |
| 9 | exponential backoff with jitter, and caps attempts so CI still fails fast when |
| 10 | the backend is genuinely down. |
| 11 | |
| 12 | This matters because `generate-toolkits.ts` issues ~6500 requests per run |
| 13 | (a few catalog pages + 3 per toolkit across a ~2.1k catalog), which can exceed |
| 14 | the backend request limit. |
| 15 | |
| 16 | ## Meta-tools reference |
| 17 | |
| 18 | `bun run generate:meta-tools` renders the reference from |
| 19 | `public/data/meta-tools.json`. Set `COMPOSIO_META_TOOLS_SCHEMA_URL` to the |
| 20 | production Platform HTTPS export to update the snapshot before rendering. The |
| 21 | export must return the same JSON array format as the snapshot. Docs - Update Data |
| 22 | reads the URL from the repository variable of the same name. With the variable |
| 23 | unset, reference pages use the checked-in schema version. |
| 24 | |
| 25 | ## Toolkit versions |
| 26 | |
| 27 | `generate-toolkits.ts` owns the complete toolkit catalog and always sources it |
| 28 | from the production API. For a version-only repair, run the narrower generator: |
| 29 | |
| 30 | ```bash |
| 31 | COMPOSIO_API_KEY=... bun run generate:toolkit-versions |
| 32 | ``` |
| 33 | |
| 34 | Both paths share `toolkit-versions.ts`, so they fetch and apply changelog values |
| 35 | with identical semantics: a toolkit missing from the production changelog gets |
| 36 | `version: null`. Any `COMPOSIO_API_BASE` override must normalize to |
| 37 | `https://backend.composio.dev/api/v3`; non-production sources fail before a |
| 38 | request is made. |
| 39 | |
| 40 | ## fetch-openapi.mjs |
| 41 | |
| 42 | Fetches the Composio OpenAPI spec and filters it for use in Fumadocs API reference documentation. |
| 43 | |
| 44 | ### Usage |
| 45 | |
| 46 | ```bash |
| 47 | bun run scripts/fetch-openapi.mjs |
| 48 | ``` |
| 49 | |
| 50 | This outputs `public/openapi.json` which is used by `lib/openapi.ts`. |
| 51 | |
| 52 | ### Why Filtering is Needed |
| 53 | |
| 54 | The raw OpenAPI spec from `https://backend.composio.dev/api/v3/openapi.json` has issues that break documentation generators: |
| 55 | |
| 56 | 1. **Endpoints with multiple tags** - Causes duplicate entries in sidebar |
| 57 | 2. **Internal endpoints exposed** - CLI, Admin, Profiling endpoints shouldn't be in public docs |
| 58 | |
| 59 | See `OPENAPI_IMPROVEMENTS.md` in the fumadocs root for planned fixes to the spec itself. |
| 60 | |
| 61 | ### What Gets Filtered |
| 62 | |
| 63 | #### Ignored Paths |
| 64 | These endpoints are completely removed: |
| 65 | - `/api/v3/mcp/validate/{uuid}` |
| 66 | - `/api/v3/cli/get-session` |
| 67 | - `/api/v3/cli/create-session` |
| 68 | - `/api/v3/auth/session/logout` |
| 69 | |
| 70 | #### Ignored Tags |
| 71 | Endpoints with only these tags are removed: |
| 72 | - `CLI` |
| 73 | - `Admin` |
| 74 | - `Profiling` |
| 75 | |
| 76 | #### Duplicate Prevention |
| 77 | If an endpoint has multiple tags, only the first tag is kept. This prevents the same endpoint appearing in multiple sidebar sections. |
| 78 | |
| 79 | ### Configuration |
| 80 | |
| 81 | The script uses environment variables: |
| 82 | |
| 83 | | Variable | Default | Description | |
| 84 | |----------|---------|-------------| |
| 85 | | `OPENAPI_SPEC_URL` | `https://backend.composio.dev/api/v3/openapi.json` | Source OpenAPI spec URL | |
| 86 | |
| 87 | For staging deployments, set: |
| 88 | ```bash |
| 89 | OPENAPI_SPEC_URL=https://staging.composio.dev/api/v3/openapi.json |
| 90 | ``` |
| 91 | |
| 92 | ## verify-kb.ts |
| 93 | |
| 94 | Checks published KB guides against the production data this repo already |
| 95 | refreshes. Runs inside `docs-update-data.yml` right after the catalog and |
| 96 | OpenAPI specs are regenerated, so freshness is verified on every production |
| 97 | deploy instead of on a review calendar. |
| 98 | |
| 99 | ```bash |
| 100 | bun run verify:kb # catalog checks only, no network |
| 101 | bun run verify:kb --check-links # also probe provider doc links |
| 102 | bun run verify:kb --json # machine-readable report |
| 103 | bun run verify:kb --markdown out.md # write the report to a file |
| 104 | ``` |
| 105 | |
| 106 | Exits non-zero when any finding is an error. `--warnings-as-errors` promotes |
| 107 | warnings too. See `docs/decisions/kb-freshness-verification.md` for what is |
| 108 | checked and why placeholders, bare URLs, and 5xx responses are excluded. |
| 109 | |
| 110 | A guide that deliberately cites a removed identifier declares it in |
| 111 | `verifyIgnoreToolSlugs` in its `kb/guides/<slug>.json`. |