Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.
Mirrored from https://github.com/Camo67/composio.git
Synced Nov 21, 09:45 UTC
docs: establish Context7 ownership and retrieval regression checks (#4872)
## Summary Addresses [DEVREL-39: Establish ownership and regression checks for Context7](https://linear.app/composio/issue/DEVREL-39/establish-ownership-and-regression-checks-for-context7). - Replace the stale “100+”/“equip's” description with count-free positioning from [production docs](https://docs.composio.dev/llms.txt): managed authentication, tool discovery, and tool execution. Public pages disagree on counts, so no new hard count is introduced. - Verify `next` is the GitHub default and is present in public retrieved source URLs. Leave `branch` omitted (Context7 defaults to the repository default), and leave **all inclusion/exclusion rules unchanged**. - Add the official `$schema` and split the overlength tool-version rule into rules below Context7's [255-character limit](https://context7.com/schema/context7.json). Preserve the v3.1 policy and clarify that scopes/required is v3.1-only. Production [REST guidance](https://docs.composio.dev/reference.md) agrees with the policy. The previous long rule was absent from returned library rules; this observation does not prove why it was omitted. - Add 15 source-backed developer questions, a dependency-free Node runner, six offline regression tests, and an opt-in manual Actions workflow that retains JSON retrieval evidence even on failure. Only offline tests join the default test command. - Add `CONTEXT7.md`: explicit human-owner TODO, claimed-settings verification checklist, manual refresh after each successful production docs deployment, and a same-suite before/after evidence gate for any future folder change. No SDK runtime, generated files, folder rules, or Linear issue state/comments changed. No Context7 login, claim, refresh request, or dashboard setting change was attempted. ## Verification Passed: - `node --test test/context7-check.test.mjs`: **6/6** tests. Covers stale facts, alternatives/whitespace, metadata/rule false positives, malformed/pending/rate-limited retrievals, continuing after errors, credential redaction, and owner-rule length limits. - `pnpm run test:context7` passed before the additional schema-limit test; the final six-test suite passed via the direct command above. - `pnpm typecheck`: **14/14 tasks** successful. - `node_modules/.bin/tsc --allowJs --checkJs --noEmit --target es2023 --module nodenext --types node --skipLibCheck scripts/context7-check.mjs test/context7-check.test.mjs`: passed against final files. - `pnpm run lint`: passed with existing warnings. JavaScript is excluded by the root lint configuration; additionally linted the new `.mjs` files with a temporary configuration inheriting root rules and clearing `ignorePatterns`: no errors, two expected console-output warnings. Temporary config removed. - `node ts/scripts/validate-agent-skills.mjs`: 19 canonical skills/guidance invariants validated. - `node ts/scripts/test-skill-routing.mjs`: 19/19 probes passed. Not green: - `pnpm run test` initially stopped because this environment lacked mise. Installed the repository-pinned mise/Node/Bun/pnpm through the existing installer. Re-running under that pinned toolchain passed `test:toolchain`, then stopped in the unchanged installer test with `FAIL: Windows guidance (missing: Windows is not supported. Use WSL)`. Remaining default-suite stages were not reached. No unrelated installer edits made. - The live retrieval baseline exits **1** because four expected-fact groups are missing, not because the runner or network failed. These are existing retrieval gaps, not proof this PR introduced a regression. - The new GitHub Actions workflow has not run; it is dispatch-only and does not refresh the index. ## Live baseline Command: `node scripts/context7-check.mjs --output /tmp/devrel-39-context7-report.json` - Public API: [GET /api/v2/context](https://context7.com/docs/api-reference/context/get-documentation-context), `libraryId=/composiohq/composio`, `type=json`, default reranking; no API key. - Started: `2026-10-14T23:57:32.102Z`; completed: `2026-10-14T23:57:52.367Z`. - Suite SHA256: `fac50cbc31f667e80e375d40db37b58243151c3dc9de10a04da4f87a38e282a9`. - Existing public index, **not refreshed for this PR**. Public sources use `blob/next`; Context7 did not supply an indexed commit SHA. The local JSON report retains full returned snippet bodies, source URLs, expected facts, timestamps, and the local config snapshot. Local config changes are not claimed to be live. ```text PASS python-install-current-sdk FAIL typescript-install-current-sdk missing [["Node.js 22.22.3"],["ESM-only"]] PASS python-initialize-api-key PASS typescript-initialize-api-key PASS project-api-key-environment FAIL auth-config-versus-connected-account missing [["COMPOSIO_MANAGE_CONNECTIONS"]] PASS python-connect-user-account PASS typescript-connect-user-account PASS python-create-session-tools PASS typescript-create-session-tools PASS execute-tools-through-session PASS select-session-toolkits PASS current-rest-api-base FAIL rest-latest-version-defaults missing [["latest toolkit version"],["GET /tools/{tool_slug}"],["POST /tools/execute/{tool_slug}/input"],["POST /tools/scopes/required"]] FAIL rest-explicit-pinned-version missing [["remains supported"]] 15 queries: 11 passed, 4 failed, 0 errors ``` Representative retrieved evidence: - TypeScript installation returned `npm install @composio/core @composio/openai-agents @openai/agents` from `README.md`, but omitted the requested runtime/ESM requirements from the [production quickstart](https://docs.composio.dev/docs/quickstart.md). - The auth question returned connected-account precedence from `docs/content/docs/configuring-sessions.mdx`, but not `COMPOSIO_MANAGE_CONNECTIONS` from the [authentication guide](https://docs.composio.dev/docs/authentication.md). - The v3.1 default question returned `curl 'https://backend.composio.dev/api/v3/tools?toolkit_slug=googledrive'` from `docs/content/docs/tools-direct/toolkit-versioning.mdx` and direct SDK version-check examples from `ts/docs/api/tools.md`. It did not retrieve the complete five-endpoint policy documented in the current [REST reference](https://docs.composio.dev/reference.md). - The pinned-version question returned `00000000_00` from the v3.1 changelog, but omitted the continued-support fact for v3. During suite development, removed incidental full-command/package-order and example-user-ID/quote assertions before freezing the suite. No expected API policy, runtime requirement, or connection-management fact was removed to hide the four remaining gaps. ## Manual owner/admin steps required 1. Assign the accountable human, backup, and Context7 teamspace in `CONTEXT7.md`. 2. Sign into Context7 and open https://context7.com/composiohq/composio/admin. If already claimed, verify the owning teamspace and your owner/admin access. If unclaimed, click **Claim Library**, copy its generated `url` and `public_key`, merge those fields into the **existing** root `context7.json` without overwriting settings, merge to `next`, then click **Claim Library** again to verify. Never use or commit the secret API key as a claim key. 3. Inspect **Configuration** and **Versions**. Confirm the effective description, branch (`next` or empty/default resolving to `next`), unchanged folders `docs/content`, `ts`, `python`, existing exclusions, REST rules, and versions. Record the date/settings evidence in this PR. Resolve non-folder discrepancies and leave folder/exclusion changes behind the comparison gate. 4. After a successful canonical production docs deploy serves the merged revision, confirm `next` already exists in Context7's branch/version configuration. Supply an owner-authorized `CONTEXT7_API_KEY` securely and trigger the documented [refresh API](https://context7.com/docs/api-reference/refresh/refresh-a-library): ```bash curl --fail-with-body --request POST \ 'https://context7.com/api/v1/refresh' \ --header "Authorization: Bearer ${CONTEXT7_API_KEY:?Set an owner-authorized Context7 key}" \ --header 'Content-Type: application/json' \ --data '{"libraryName":"/composiohq/composio","branch":"next"}' ``` 5. A 200 means refresh started, not finished. Wait for completion, confirm the indexed branch/revision if exposed, record any unavailable revision explicitly, then rerun the baseline or dispatch **Context7 Retrieval Check**. Attach the resulting report and review every FAIL/ERROR. 6. For automatic post-deploy refreshes, first identify the canonical deployment success signal and its production revision. Observed `Production` deployments reported preview hostnames or empty environment URLs, so this PR does **not** wire a potentially wrong automatic trigger. Until that signal is verified, the owner performs step 4 after each successful production docs deploy. Preview, failed deploy, and push events are not refresh triggers. ## Acceptance checklist - **Partial:** Configuration and claimed-library settings reflect production docs. Repository description/rules/schema checked and updated; effective claimed settings and published metadata require authenticated human verification and refresh. - **Done (documentation):** Accountable owner placeholder and refresh trigger/how-to documented. Owner TODO is explicitly allowed by the issue; no person has been assigned automatically. Refresh is a manual after-success policy, not deployed automation. - **Done:** Repeatable fixed query set detects missing retrieval facts. Baseline above records 11 PASS / 4 FAIL / 0 ERROR; offline behavior tests pass. - **Done / not applicable:** Folder-rule changes require before/after evidence. No include/exclude changes in this PR; the controlled comparison procedure is documented. Designated reviewer: @jkomyno. Please review the full diff before submitting/merging. No changeset: no published TypeScript package changed.