Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.
docs: add agent-readable discovery and freshness metadata (#4874)
## DEVREL-36 https://linear.app/composio/issue/DEVREL-36/add-discovery-caching-and-freshness-metadata-to-agent-readable-docs HTML pages now advertise their existing Markdown representations. Negotiated responses preserve `Vary: Accept`, Markdown includes machine-readable metadata, and missing Markdown pages return discovery links with HTTP 404. ### Changes - Add canonical and Markdown URLs, docs product, REST version, and `last_updated` to per-page Markdown front matter. - Generate a persisted freshness manifest for authored pages and dated changelog entries. Use explicit `last_updated` or Git source history, exclude shallow-boundary guesses, and never use build time or filesystem mtime. Deployed routes do not need Git. - Return `last_updated: null` and omit sitemap `lastmod` for generated API/toolkit pages, KB guides, and browse pages. Changes to shared specifications, catalogs, or manifests must not make unchanged pages appear newly updated. Add coverage only when changes can be attributed to each page. - Preserve redirect precedence and avoid advertising Markdown for HTML-only pages. - Retain the two-line Next.js 16.3.8 dependency patch that preserves middleware variance during HTML rendering. [Upstream issue #85852](https://github.com/vercel/next.js/issues/85852) tracks the regression. Remove the patch once an unpatched production build passes the live negotiation test. ### Validation of the freshness policy update From `docs/`: - `bun run test`: 727 passed; one test was blocked by sandbox restrictions on local HTTP listeners. Rerunning `bun test tests/static/kb-query-analytics.test.ts` with listener access passed all 14 tests in that file. - `bun run types:check`: passed. - `bun run lint`: passed with existing warnings. - `bun run lint:links`: passed, 0 errors. - `bun run build --experimental-build-mode compile`: passed. - `TEST_BASE_URL=http://localhost:3036 bun test tests/integration/agent-metadata.test.ts tests/integration/markdown-discovery.test.ts --timeout 30000`: 33 passed against the compiled production server. Includes null Markdown freshness and absent sitemap dates for generated pages, KB guides, and browse pages, plus discovery and cache variance regressions. Repository agent guidance and skill-routing validators passed. `git diff --check` passed. The VM disk was nearly full, so local production output was placed in shared memory. Its generated module symlink was adjusted for that temporary location, and `NODE_PATH` pointed to the docs dependencies. No tracked build configuration changed. A full prerender build was not rerun locally; the previous revision passed docs CI and Vercel deployment, and CI is rerunning for this update. ### Vercel history fix Vercel's shallow checkout left older authored pages without dates. The build command now runs `scripts/vercel-build.sh`, which fetches missing history before `bun run build`. Complete clones skip the fetch. A failed fetch stops the build. Validation: all 7 content-freshness tests passed, including a local depth-1 clone that gains its original source timestamp after the build step. The test also covers complete clones and fetch failures. Focused lint, shell syntax validation, and `git diff --check` passed. The updated Vercel deployment has not yet been verified. ### Limits Source history records edits, not verification that guidance remains current. Unknown freshness stays null or absent. Full Git history is needed for complete authored-page coverage. Dated changelog pages use the latest known update among entries for that date. Please review the full diff, especially the Next.js dependency patch and freshness exclusions, before merging.
No changes.