@devries-cameron20 / composio Mirror · read-only / commits / 7bd3e0669
Fork 0

Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

Mirrored from https://github.com/Camo67/composio.git

Synced Nov 21, 21:00 UTC

docs: migrate search to Algolia (#3452)

## Summary - Switch docs search dialog to Algolia when public Algolia env vars are configured, with local `/api/search` fallback for development/tests - Add a shared docs search index builder plus `bun run sync:search` to publish records to Algolia - Add a GitHub Actions workflow to dry-run and sync the Algolia index on `next` docs changes when secrets are configured - Document required Algolia env vars and sync command ## Tests - `cd docs && bun run types:check` - `cd docs && bun run sync:search --dry-run` - `cd docs && bun test tests/static/` - `cd docs && bunx eslint components/custom-search-dialog.tsx lib/search-index.ts scripts/sync-algolia-search.ts` - `cd docs && bun run build` ## Notes - `bun run lint` still fails on existing unrelated repo-wide lint errors in files outside this change (for example `app/global-error.tsx`, `components/ask-ai-button.tsx`, `components/version-selector.tsx`). The changed search files pass targeted ESLint. - Production search requires `NEXT_PUBLIC_ALGOLIA_APP_ID`, `NEXT_PUBLIC_ALGOLIA_SEARCH_API_KEY`, and optionally `NEXT_PUBLIC_ALGOLIA_INDEX_NAME` (default `composio_docs`). Index syncing requires `ALGOLIA_APP_ID`/`ALGOLIA_ADMIN_API_KEY` secrets.

Rahul Tarak committed May 19, 2026 at 01:33 UTC 7bd3e0669cebdc142a302e5423a1b31bd3b76504
9 files changed +584 -93
.github/workflows/docs-search-sync.yml new
+46
@@ -0,0 +1,46 @@
1 +name: Docs - Sync Algolia Search
2 +
3 +on:
4 + push:
5 + branches:
6 + - next
7 + paths:
8 + - 'docs/**'
9 + workflow_dispatch:
10 +
11 +permissions:
12 + contents: read
13 +
14 +jobs:
15 + sync-search:
16 + runs-on: ubuntu-latest
17 + defaults:
18 + run:
19 + working-directory: ./docs
20 + env:
21 + ALGOLIA_APP_ID: ${{ vars.ALGOLIA_APP_ID || '62HI9PQZ1L' }}
22 + ALGOLIA_ADMIN_API_KEY: ${{ secrets.ALGOLIA_ADMIN_API_KEY }}
23 + ALGOLIA_INDEX_NAME: ${{ vars.ALGOLIA_INDEX_NAME || 'docs_composio_dev_62hi9pqz1l_pages' }}
24 +
25 + steps:
26 + - name: Checkout repository
27 + uses: actions/checkout@v6
28 +
29 + - name: Setup Bun
30 + uses: oven-sh/setup-bun@v2
31 + with:
32 + bun-version-file: '.bun-version'
33 +
34 + - name: Install dependencies
35 + run: bun install
36 +
37 + - name: Validate search index generation
38 + run: bun run sync:search --dry-run
39 +
40 + - name: Sync Algolia search index
41 + if: env.ALGOLIA_APP_ID != '' && env.ALGOLIA_ADMIN_API_KEY != ''
42 + run: bun run sync:search
43 +
44 + - name: Skip sync without Algolia secrets
45 + if: env.ALGOLIA_APP_ID == '' || env.ALGOLIA_ADMIN_API_KEY == ''
46 + run: echo "Skipping Algolia sync because ALGOLIA_APP_ID or ALGOLIA_ADMIN_API_KEY is not configured."
docs/CLAUDE.md
+7
@@ -9,6 +9,7 @@ bun install # Install dependencies
9 bun run dev # Dev server (http://localhost:3000)
10 bun run build # Production build (validates TS code blocks)
11 bun run types:check # Type check
12 +bun run sync:search # Sync Algolia docs search index (requires Algolia env vars)
13 ```
14
15 ## Project Structure
@@ -83,6 +84,12 @@ Detailed documentation for Claude is organized in `.claude/`:
84
85 `content/docs/glossary.mdx` defines key Composio terms (auth config, session, toolkit, etc.) using `<Glossary>` and `<GlossaryTerm name="...">` components (`components/glossary.tsx`). The component renders a filterable two-column table. The markdown converter in `lib/source.ts` converts `<GlossaryTerm>` tags to `### Term` headings for LLM-friendly output. When adding new Composio concepts, add a `<GlossaryTerm>` entry and update the `keywords` frontmatter array.
86
87 +## Search
88 +
89 +The docs search dialog uses Algolia in production when `NEXT_PUBLIC_ALGOLIA_SEARCH_API_KEY` is configured. `NEXT_PUBLIC_ALGOLIA_APP_ID` defaults to `62HI9PQZ1L`, and `NEXT_PUBLIC_ALGOLIA_INDEX_NAME` defaults to `docs_composio_dev_62hi9pqz1l_pages`. Without a search API key it falls back to `/api/search` for local development and tests. Algolia searches request `clickAnalytics`, and the client sends search result view/click events with `search-insights`.
90 +
91 +Run `ALGOLIA_APP_ID=62HI9PQZ1L ALGOLIA_ADMIN_API_KEY=... ALGOLIA_INDEX_NAME=docs_composio_dev_62hi9pqz1l_pages bun run sync:search` from `docs/` to rebuild the Algolia index.
92 +
93 ## API Versioning
94
95 The API reference supports v3.1 (latest, default) and v3.0 side-by-side:
docs/README.md
+19
@@ -74,6 +74,24 @@ All TypeScript code blocks in MDX files are **type-checked at build time** using
74
75 See `CLAUDE.md` for detailed patterns and troubleshooting.
76
77 +## Search
78 +
79 +Docs search uses Algolia when `NEXT_PUBLIC_ALGOLIA_SEARCH_API_KEY` is set; otherwise it falls back to the local Fumadocs `/api/search` endpoint for development and tests. The Algolia client requests `clickAnalytics` and sends search result view/click events with `search-insights`.
80 +
81 +```bash
82 +NEXT_PUBLIC_ALGOLIA_APP_ID=62HI9PQZ1L # optional; default shown
83 +NEXT_PUBLIC_ALGOLIA_SEARCH_API_KEY=...
84 +NEXT_PUBLIC_ALGOLIA_INDEX_NAME=docs_composio_dev_62hi9pqz1l_pages # optional; default shown
85 +```
86 +
87 +Sync the Algolia index with an admin key:
88 +
89 +```bash
90 +ALGOLIA_APP_ID=62HI9PQZ1L
91 +ALGOLIA_ADMIN_API_KEY=...
92 +ALGOLIA_INDEX_NAME=docs_composio_dev_62hi9pqz1l_pages bun run sync:search
93 +```
94 +
95 ## Commands
96
97 | Command | Description |
@@ -81,4 +99,5 @@ See `CLAUDE.md` for detailed patterns and troubleshooting.
99 | `bun run dev` | Dev server |
100 | `bun run build` | Production build (validates TS code blocks) |
101 | `bun run types:check` | Type check |
102 +| `bun run sync:search` | Sync docs search records to Algolia |
103
docs/app/api/search/route.ts
+4 -85
@@ -1,89 +1,8 @@
1 -// Use direct imports from collections to avoid top-level await in lib/source.ts
2 -import { docs, reference, cookbooks, toolkits, changelog } from 'fumadocs-mdx:collections/server';
1 import { createSearchAPI } from 'fumadocs-core/search/server';
4 -import { loader, multiple } from 'fumadocs-core/source';
5 -import { lucideIconsPlugin } from 'fumadocs-core/source/lucide-icons';
6 -import { openapiSource, openapiPlugin } from 'fumadocs-openapi/server';
7 -import { openapi } from '@/lib/openapi';
8 -import { getAllToolkitsSync } from '@/lib/toolkit-data';
2 +import { getDocsSearchIndexes } from '@/lib/search-index';
3
10 -// Create loaders directly here to avoid the problematic lib/source.ts import
11 -const docsSource = loader({
12 - baseUrl: '/docs',
13 - source: docs.toFumadocsSource(),
14 - plugins: [lucideIconsPlugin()],
15 -});
16 -
17 -const cookbooksSource = loader({
18 - baseUrl: '/cookbooks',
19 - source: cookbooks.toFumadocsSource(),
20 - plugins: [lucideIconsPlugin()],
21 -});
22 -
23 -const toolkitsSource = loader({
24 - baseUrl: '/toolkits',
25 - source: toolkits.toFumadocsSource(),
26 - plugins: [lucideIconsPlugin()],
27 -});
28 -
29 -// Dynamic toolkit entries from toolkits.json
30 -const mdxToolkitSlugs = new Set(
31 - toolkitsSource.getPages().map((page) => page.slugs.join('/')),
32 -);
33 -
34 -const dynamicToolkitIndexes = getAllToolkitsSync()
35 - .filter((toolkit) => !mdxToolkitSlugs.has(toolkit.slug))
36 - .map((toolkit) => ({
37 - id: `/toolkits/${toolkit.slug}`,
38 - title: toolkit.name,
39 - description: toolkit.description,
40 - url: `/toolkits/${toolkit.slug}`,
41 - structuredData: { headings: [] as never[], contents: [] as never[] },
42 - keywords: [toolkit.slug, toolkit.category].filter(Boolean) as string[],
43 - }));
44 -
45 -// Changelog entries
46 -const changelogIndexes = changelog.map((entry) => ({
47 - id: `/docs/changelog/${entry.date.replace(/-/g, '/')}#${entry.title}`,
48 - title: entry.title,
49 - description: entry.description ?? '',
50 - url: `/docs/changelog/${entry.date.replace(/-/g, '/')}`,
51 - structuredData: { headings: [] as never[], contents: [] as never[] },
52 - keywords: ['changelog'],
53 -}));
54 -
55 -// Use dynamic indexes to support async OpenAPI page loading
4 +// Fallback/local search endpoint. Production search uses Algolia from the client
5 +// when NEXT_PUBLIC_ALGOLIA_* variables are configured.
6 export const { GET } = createSearchAPI('advanced', {
57 - indexes: async () => {
58 - // Load OpenAPI pages and build full reference source
59 - const openapiPages = await openapiSource(openapi, {
60 - groupBy: 'tag',
61 - baseDir: 'api-reference',
62 - });
63 -
64 - const fullReferenceSource = loader({
65 - baseUrl: '/reference',
66 - source: multiple({
67 - mdx: reference.toFumadocsSource(),
68 - openapi: openapiPages,
69 - }),
70 - plugins: [lucideIconsPlugin(), openapiPlugin()],
71 - });
72 -
73 - const mdxIndexes = [
74 - ...docsSource.getPages(),
75 - ...cookbooksSource.getPages(),
76 - ...toolkitsSource.getPages(),
77 - ...fullReferenceSource.getPages(),
78 - ].map((page) => ({
79 - id: page.url,
80 - title: page.data.title ?? 'Untitled',
81 - description: page.data.description,
82 - url: page.url,
83 - structuredData: page.data.structuredData,
84 - keywords: 'keywords' in page.data ? (page.data.keywords as string[]) : undefined,
85 - }));
86 -
87 - return [...mdxIndexes, ...dynamicToolkitIndexes, ...changelogIndexes];
88 - },
7 + indexes: getDocsSearchIndexes,
8 });
docs/bun.lock
+35
@@ -1,5 +1,6 @@
1 {
2 "lockfileVersion": 1,
3 + "configVersion": 0,
4 "workspaces": {
5 "": {
6 "name": "fumadocs",
@@ -7,6 +8,7 @@
8 "@radix-ui/react-tabs": "^1.1.13",
9 "@shikijs/twoslash": "^3.21.0",
10 "@vercel/analytics": "^1.6.1",
11 + "algoliasearch": "^5",
12 "class-variance-authority": "^0.7.1",
13 "clsx": "^2.1.1",
14 "fumadocs-core": "^16.4.2",
@@ -19,6 +21,7 @@
21 "posthog-js": "^1.319.0",
22 "react": "^19.2.3",
23 "react-dom": "^19.2.3",
24 + "search-insights": "^2",
25 "shiki": "^3.20.0",
26 "tailwind-merge": "^3.4.0",
27 },
@@ -95,6 +98,34 @@
98
99 "@ai-sdk/ui-utils-v5": ["@ai-sdk/ui-utils@1.2.11", "", { "dependencies": { "@ai-sdk/provider": "1.1.3", "@ai-sdk/provider-utils": "2.2.8", "zod-to-json-schema": "^3.24.1" }, "peerDependencies": { "zod": "^3.23.8" } }, "sha512-3zcwCc8ezzFlwp3ZD15wAPjf2Au4s3vAbKsXQVyhxODHcmu0iyPO2Eua6D/vicq/AUm/BAo60r97O6HU+EI0+w=="],
100
101 + "@algolia/abtesting": ["@algolia/abtesting@1.18.1", "", { "dependencies": { "@algolia/client-common": "5.52.1", "@algolia/requester-browser-xhr": "5.52.1", "@algolia/requester-fetch": "5.52.1", "@algolia/requester-node-http": "5.52.1" } }, "sha512-aehCadlWOGvrT91KUIZpC0MbB8KBW9yUuvTJFd2xesR7le/IsT4nJUnjCCZ4ZqZCeTcPHPV5mo//fZ5oxcSVYw=="],
102 +
103 + "@algolia/client-abtesting": ["@algolia/client-abtesting@5.52.1", "", { "dependencies": { "@algolia/client-common": "5.52.1", "@algolia/requester-browser-xhr": "5.52.1", "@algolia/requester-fetch": "5.52.1", "@algolia/requester-node-http": "5.52.1" } }, "sha512-HmXOGBOAOJPounpBzBpuY0zDYeiCpxgHnQmuA7JO6ScukcBdGp3/XM9zJk5pJx/xNGD68mbPGXWpDxGtl6BwDQ=="],
104 +
105 + "@algolia/client-analytics": ["@algolia/client-analytics@5.52.1", "", { "dependencies": { "@algolia/client-common": "5.52.1", "@algolia/requester-browser-xhr": "5.52.1", "@algolia/requester-fetch": "5.52.1", "@algolia/requester-node-http": "5.52.1" } }, "sha512-5oo4+I8iixie9vXhCyNFCzeIr8pqA3FQ//VsLHTDvZAV4ttYOPGvYHGQq5NSalrLx5Jc3dRro/5uDOlnUMcBJg=="],
106 +
107 + "@algolia/client-common": ["@algolia/client-common@5.52.1", "", {}, "sha512-qCDoZfx5MpX7XQzvQ3bC4tSEMkQWQMaF/ABtLuoze03Y/flR563CCSws02qIJ23oX7lxl92LsilZjINVyTdtLw=="],
108 +
109 + "@algolia/client-insights": ["@algolia/client-insights@5.52.1", "", { "dependencies": { "@algolia/client-common": "5.52.1", "@algolia/requester-browser-xhr": "5.52.1", "@algolia/requester-fetch": "5.52.1", "@algolia/requester-node-http": "5.52.1" } }, "sha512-hnGs0/lsFJ2PWDxNBz7pxreXo/Xz7gxYRcfePBUjsH26ad0kU/sgnVZd9LwWBpsQv65z2jlb5dkyaB9WE9M9FQ=="],
110 +
111 + "@algolia/client-personalization": ["@algolia/client-personalization@5.52.1", "", { "dependencies": { "@algolia/client-common": "5.52.1", "@algolia/requester-browser-xhr": "5.52.1", "@algolia/requester-fetch": "5.52.1", "@algolia/requester-node-http": "5.52.1" } }, "sha512-2VxxNc/uBysyKvGeBdSM5n9eIDKH8kWD7wd9/yqbJAiVwU4Yv6tU1LSJusHKrXV/aCu1KW7t9Gug9QyeEmtn/Q=="],
112 +
113 + "@algolia/client-query-suggestions": ["@algolia/client-query-suggestions@5.52.1", "", { "dependencies": { "@algolia/client-common": "5.52.1", "@algolia/requester-browser-xhr": "5.52.1", "@algolia/requester-fetch": "5.52.1", "@algolia/requester-node-http": "5.52.1" } }, "sha512-O6mPtsw3xEfNOe6gWFpYLeAZAIljNa4Hgna3bq15PwyN7nbjTY0wXJFRbzs/0YVf75Br+SbOQUmjKxXYjDiSiQ=="],
114 +
115 + "@algolia/client-search": ["@algolia/client-search@5.52.1", "", { "dependencies": { "@algolia/client-common": "5.52.1", "@algolia/requester-browser-xhr": "5.52.1", "@algolia/requester-fetch": "5.52.1", "@algolia/requester-node-http": "5.52.1" } }, "sha512-gA8oJOV1LnQQkDf91iebNnFInHuW0gRPEgLSOQ7EfipCEjYTHm5swm1DlH9H5RaRw4RrHuzHBegnlzc0MAstcg=="],
116 +
117 + "@algolia/ingestion": ["@algolia/ingestion@1.52.1", "", { "dependencies": { "@algolia/client-common": "5.52.1", "@algolia/requester-browser-xhr": "5.52.1", "@algolia/requester-fetch": "5.52.1", "@algolia/requester-node-http": "5.52.1" } }, "sha512-U9zZfc5xIu9wRxZkt+HceJUAD4VKHKbAyLSloJdEyMRmphXeibfrY9cxqIXBcmPeZzGhn3Imb35Dq8l19PkJhw=="],
118 +
119 + "@algolia/monitoring": ["@algolia/monitoring@1.52.1", "", { "dependencies": { "@algolia/client-common": "5.52.1", "@algolia/requester-browser-xhr": "5.52.1", "@algolia/requester-fetch": "5.52.1", "@algolia/requester-node-http": "5.52.1" } }, "sha512-a3SGNceHmkQfq77iG8Ka+w1pvwfZa/0lzEIgse30fL0kD+yKnd/dg0dQvSfFPAEt2f21DMcGkDSSeJlO3KdQjQ=="],
120 +
121 + "@algolia/recommend": ["@algolia/recommend@5.52.1", "", { "dependencies": { "@algolia/client-common": "5.52.1", "@algolia/requester-browser-xhr": "5.52.1", "@algolia/requester-fetch": "5.52.1", "@algolia/requester-node-http": "5.52.1" } }, "sha512-z98QEguCFDpxb4S/PyrUK1igqF8tPsdbqOUUO6ON91vJ58w+Gwa6ncrI0oNXSFcrkxA5EqPKPQ2A1PBCn08TYQ=="],
122 +
123 + "@algolia/requester-browser-xhr": ["@algolia/requester-browser-xhr@5.52.1", "", { "dependencies": { "@algolia/client-common": "5.52.1" } }, "sha512-CI7+/0I11QeZM59Uc8whd2or0kqzFVjpaPn9Qpwll/krHcBAxk24WkAQ6WX+IwDVMfpont4YGbKwAmCre3vE8Q=="],
124 +
125 + "@algolia/requester-fetch": ["@algolia/requester-fetch@5.52.1", "", { "dependencies": { "@algolia/client-common": "5.52.1" } }, "sha512-S6bDuw9byfOvm3T71cgdoZgrgnZq6hpdMLkx52Louh57nUAmvGQESz2aojOynQHjbTiV55smvAFbgn0qT4tJrg=="],
126 +
127 + "@algolia/requester-node-http": ["@algolia/requester-node-http@5.52.1", "", { "dependencies": { "@algolia/client-common": "5.52.1" } }, "sha512-tqZXM+54rWo4mk5jL5Z/flE11nPmNEdXwFBM5py9DkOmbjeCNemfVd45FyM97XdzfZ0dl9uOJC6PYn1FpkeyQg=="],
128 +
129 "@alloc/quick-lru": ["@alloc/quick-lru@5.2.0", "", {}, "sha512-UrcABB+4bUrFABwbluTIBErXwvbsU/V7TZWfmbgJfbkwiBuziS9gxdODUyuiecfdGQ85jglMW6juS3+z5TsKLw=="],
130
131 "@antfu/install-pkg": ["@antfu/install-pkg@1.1.0", "", { "dependencies": { "package-manager-detector": "^1.3.0", "tinyexec": "^1.0.1" } }, "sha512-MGQsmw10ZyI+EJo45CdSER4zEb+p31LpDAFp2Z3gkSd1yqVZGi0Ebx++YTEMonJy4oChEMLsxZ64j8FH6sSqtQ=="],
@@ -865,6 +896,8 @@
896
897 "ajv-formats": ["ajv-formats@3.0.1", "", { "dependencies": { "ajv": "^8.0.0" } }, "sha512-8iUql50EUR+uUcdRQ3HDqa6EVyo3docL8g5WJ3FNcWmu62IbkGUue/pEyLBW8VGKKucTPgqeks4fIU1DA4yowQ=="],
898
899 + "algoliasearch": ["algoliasearch@5.52.1", "", { "dependencies": { "@algolia/abtesting": "1.18.1", "@algolia/client-abtesting": "5.52.1", "@algolia/client-analytics": "5.52.1", "@algolia/client-common": "5.52.1", "@algolia/client-insights": "5.52.1", "@algolia/client-personalization": "5.52.1", "@algolia/client-query-suggestions": "5.52.1", "@algolia/client-search": "5.52.1", "@algolia/ingestion": "1.52.1", "@algolia/monitoring": "1.52.1", "@algolia/recommend": "5.52.1", "@algolia/requester-browser-xhr": "5.52.1", "@algolia/requester-fetch": "5.52.1", "@algolia/requester-node-http": "5.52.1" } }, "sha512-fHA8+kXTbjagw3jkLiaS7KKrH8qe2DyOsiUhGlN4cdT77PEsfqXZl7ewDk1hsg+pJnPlnE50XtLxjR91iJOpmg=="],
900 +
901 "alien-signals": ["alien-signals@3.1.2", "", {}, "sha512-d9dYqZTS90WLiU0I5c6DHj/HcKkF8ZyGN3G5x8wSbslulz70KOxaqCT0hQCo9KOyhVqzqGojvNdJXoTumZOtcw=="],
902
903 "ansi-regex": ["ansi-regex@6.2.2", "", {}, "sha512-Bq3SmSpyFHaWjPk8If9yc6svM8c56dB5BAtW4Qbw5jHTwwXXcTLoRMkpDJp6VL0XzlWaCHTXrkFURMYmD0sLqg=="],
@@ -1999,6 +2032,8 @@
2032
2033 "scroll-into-view-if-needed": ["scroll-into-view-if-needed@3.1.0", "", { "dependencies": { "compute-scroll-into-view": "^3.0.2" } }, "sha512-49oNpRjWRvnU8NyGVmUaYG4jtTkNonFZI86MmGRDqBphEK2EXT9gdEUoQPZhuBM8yWHxCWbobltqYO5M4XrUvQ=="],
2034
2035 + "search-insights": ["search-insights@2.17.3", "", {}, "sha512-RQPdCYTa8A68uM2jwxoY842xDhvx3E5LFL1LxvxCNMev4o5mLuokczhzjAgGwUZBAmOKZknArSxLKmXtIi2AxQ=="],
2036 +
2037 "section-matter": ["section-matter@1.0.0", "", { "dependencies": { "extend-shallow": "^2.0.1", "kind-of": "^6.0.0" } }, "sha512-vfD3pmTzGpufjScBh50YHKzEu2lxBWhVEHsNGoEXmCmn2hKGfeNLYMzCJpe8cD7gqX7TJluOVpBkAequ6dgMmA=="],
2038
2039 "secure-json-parse": ["secure-json-parse@2.7.0", "", {}, "sha512-6aU+Rwsezw7VR8/nyvKTx8QpWH9FrcYiXXlqC4z5d5XQBDRqtbfsRjnwGyqbi3gddNtWHuEk9OANUotL26qKUw=="],
docs/components/custom-search-dialog.tsx
+147 -8
@@ -1,7 +1,9 @@
1 'use client';
2
3 -import { useEffect, useState } from 'react';
3 +import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
4 import Link from 'next/link';
5 +import { liteClient } from 'algoliasearch/lite';
6 +import aa from 'search-insights';
7 import {
8 SearchDialog,
9 SearchDialogClose,
@@ -15,14 +17,20 @@ import {
17 } from 'fumadocs-ui/components/dialog/search';
18 import { useI18n } from 'fumadocs-ui/contexts/i18n';
19 import { useDocsSearch } from 'fumadocs-core/search/client';
20 +import type { BaseIndex } from 'fumadocs-core/search/algolia';
21 import { BotMessageSquare } from 'lucide-react';
22 import { toggleDecimalWidget, detectMac } from './ask-ai-button';
23
24 function MetaKey() {
25 const [key, setKey] = useState('⌘');
26 +
27 useEffect(() => {
24 - if (!detectMac()) setKey('Ctrl');
28 + if (detectMac()) return;
29 +
30 + const id = window.setTimeout(() => setKey('Ctrl'), 0);
31 + return () => window.clearTimeout(id);
32 }, []);
33 +
34 return key;
35 }
36
@@ -37,17 +45,136 @@ interface CustomSearchDialogProps extends SharedProps {
45 api?: string;
46 }
47
48 +type AlgoliaHit = {
49 + objectID: string;
50 + url?: string;
51 +};
52 +
53 +type AlgoliaSearchResponse = {
54 + hits: AlgoliaHit[];
55 + queryID?: string;
56 +};
57 +
58 +type AlgoliaHitMeta = {
59 + objectID: string;
60 + position: number;
61 + queryID?: string;
62 +};
63 +
64 export default function CustomSearchDialog({
65 defaultLinks = [],
66 api = '/api/search',
67 ...props
68 }: CustomSearchDialogProps) {
69 const { locale } = useI18n();
46 - const { search, setSearch, query } = useDocsSearch({
47 - type: 'fetch',
48 - locale,
49 - api,
50 - });
70 + const algoliaHitMetaRef = useRef(new Map<string, AlgoliaHitMeta>());
71 + const insightsInitializedRef = useRef(false);
72 +
73 + const ensureAlgoliaInsights = useCallback((appId: string, searchApiKey: string) => {
74 + if (insightsInitializedRef.current) return;
75 +
76 + aa('init', {
77 + appId,
78 + apiKey: searchApiKey,
79 + useCookie: true,
80 + });
81 + insightsInitializedRef.current = true;
82 + }, []);
83 +
84 + const clientOptions = useMemo(() => {
85 + const appId = process.env.NEXT_PUBLIC_ALGOLIA_APP_ID ?? '62HI9PQZ1L';
86 + const searchApiKey = process.env.NEXT_PUBLIC_ALGOLIA_SEARCH_API_KEY;
87 + const indexName = process.env.NEXT_PUBLIC_ALGOLIA_INDEX_NAME ?? 'docs_composio_dev_62hi9pqz1l_pages';
88 +
89 + if (appId && searchApiKey) {
90 + const client = liteClient(appId, searchApiKey);
91 +
92 + return {
93 + type: 'algolia' as const,
94 + client,
95 + indexName,
96 + locale,
97 + onSearch: async (query: string, tag?: string) => {
98 + ensureAlgoliaInsights(appId, searchApiKey);
99 +
100 + const result = await client.searchForHits<BaseIndex>({
101 + requests: [
102 + {
103 + type: 'default',
104 + indexName,
105 + query,
106 + distinct: 5,
107 + hitsPerPage: 10,
108 + filters: tag ? `tag:${tag}` : undefined,
109 + clickAnalytics: true,
110 + },
111 + ],
112 + });
113 + const response = result.results[0] as AlgoliaSearchResponse;
114 + const metadata = new Map<string, AlgoliaHitMeta>();
115 +
116 + response.hits.forEach((hit, index) => {
117 + if (!hit.url || metadata.has(hit.url)) return;
118 + metadata.set(hit.url, {
119 + objectID: hit.objectID,
120 + position: index + 1,
121 + queryID: response.queryID,
122 + });
123 + });
124 +
125 + algoliaHitMetaRef.current = metadata;
126 +
127 + const viewedObjectIDs = Array.from(metadata.values())
128 + .slice(0, 20)
129 + .map((hit) => hit.objectID);
130 +
131 + if (viewedObjectIDs.length > 0) {
132 + aa('viewedObjectIDs', {
133 + index: indexName,
134 + eventName: 'Docs Search Results Viewed',
135 + objectIDs: viewedObjectIDs,
136 + });
137 + }
138 +
139 + return result;
140 + },
141 + };
142 + }
143 +
144 + return {
145 + type: 'fetch' as const,
146 + locale,
147 + api,
148 + };
149 + }, [api, ensureAlgoliaInsights, locale]);
150 +
151 + const { search, setSearch, query } = useDocsSearch(clientOptions);
152 +
153 + const trackAlgoliaClick = useCallback((href: string) => {
154 + const url = new URL(href, window.location.origin);
155 + const path = `${url.pathname}${url.hash}`;
156 + const hit = algoliaHitMetaRef.current.get(path) ?? algoliaHitMetaRef.current.get(url.pathname);
157 + const indexName = process.env.NEXT_PUBLIC_ALGOLIA_INDEX_NAME ?? 'docs_composio_dev_62hi9pqz1l_pages';
158 +
159 + if (!hit) return;
160 +
161 + if (hit.queryID) {
162 + aa('clickedObjectIDsAfterSearch', {
163 + index: indexName,
164 + eventName: 'Docs Search Result Clicked',
165 + queryID: hit.queryID,
166 + objectIDs: [hit.objectID],
167 + positions: [hit.position],
168 + });
169 + return;
170 + }
171 +
172 + aa('clickedObjectIDs', {
173 + index: indexName,
174 + eventName: 'Docs Search Result Clicked',
175 + objectIDs: [hit.objectID],
176 + });
177 + }, []);
178
179 return (
180 <SearchDialog
@@ -63,7 +190,19 @@ export default function CustomSearchDialog({
190 <SearchDialogInput />
191 <SearchDialogClose />
192 </SearchDialogHeader>
66 - <div className="search-scroll-container max-md:min-h-0 max-md:flex-1 max-md:overflow-y-auto">
193 + <div
194 + className="search-scroll-container max-md:min-h-0 max-md:flex-1 max-md:overflow-y-auto"
195 + onClickCapture={(event) => {
196 + const target = event.target;
197 + if (!(target instanceof Element)) return;
198 +
199 + const anchor = target.closest('a[href]');
200 + const href = anchor?.getAttribute('href');
201 + if (!href) return;
202 +
203 + trackAlgoliaClick(href);
204 + }}
205 + >
206 {query.data === 'empty' && defaultLinks.length > 0 ? (
207 <div className="flex flex-col p-2">
208 {defaultLinks.map((link) => (
docs/lib/search-index.ts new
+283
@@ -0,0 +1,283 @@
1 +// Use direct imports from collections to avoid top-level await in lib/source.ts.
2 +import { existsSync, readdirSync, readFileSync } from 'node:fs';
3 +import { join, relative } from 'node:path';
4 +import { docs, reference, cookbooks, toolkits } from 'fumadocs-mdx:collections/server';
5 +import { loader, multiple } from 'fumadocs-core/source';
6 +import { lucideIconsPlugin } from 'fumadocs-core/source/lucide-icons';
7 +import { openapiSource, openapiPlugin } from 'fumadocs-openapi/server';
8 +import type { AdvancedIndex } from 'fumadocs-core/search/server';
9 +import type { DocumentRecord } from 'fumadocs-core/search/algolia';
10 +import { openapi } from '@/lib/openapi';
11 +import { getAllToolkitsSync } from '@/lib/toolkit-data';
12 +
13 +// Create loaders directly here to avoid the problematic lib/source.ts import.
14 +const docsSource = loader({
15 + baseUrl: '/docs',
16 + source: docs.toFumadocsSource(),
17 + plugins: [lucideIconsPlugin()],
18 +});
19 +
20 +const cookbooksSource = loader({
21 + baseUrl: '/cookbooks',
22 + source: cookbooks.toFumadocsSource(),
23 + plugins: [lucideIconsPlugin()],
24 +});
25 +
26 +const toolkitsSource = loader({
27 + baseUrl: '/toolkits',
28 + source: toolkits.toFumadocsSource(),
29 + plugins: [lucideIconsPlugin()],
30 +});
31 +
32 +// Dynamic toolkit entries from toolkits.json.
33 +const mdxToolkitSlugs = new Set(
34 + toolkitsSource.getPages().map((page) => page.slugs.join('/')),
35 +);
36 +
37 +const dynamicToolkitIndexes = getAllToolkitsSync()
38 + .filter((toolkit) => !mdxToolkitSlugs.has(toolkit.slug))
39 + .map((toolkit) => ({
40 + id: `/toolkits/${toolkit.slug}`,
41 + title: toolkit.name,
42 + description: toolkit.description,
43 + url: `/toolkits/${toolkit.slug}`,
44 + structuredData: { headings: [], contents: [] },
45 + keywords: [toolkit.slug, toolkit.category].filter(Boolean) as string[],
46 + } satisfies SearchIndex));
47 +
48 +function getFrontmatterValue(frontmatter: string, key: string): string | undefined {
49 + const match = frontmatter.match(new RegExp(`^${key}:\\s*(?:"([^"]*)"|'([^']*)'|(.+))$`, 'm'));
50 + const value = match?.[1] ?? match?.[2] ?? match?.[3];
51 + return value?.trim();
52 +}
53 +
54 +function getChangelogIndexes(): SearchIndex[] {
55 + const changelogDir = join(process.cwd(), 'content/changelog');
56 + if (!existsSync(changelogDir)) return [];
57 +
58 + return readdirSync(changelogDir)
59 + .filter((file) => file.endsWith('.mdx'))
60 + .flatMap((file) => {
61 + const source = readFileSync(join(changelogDir, file), 'utf8');
62 + const frontmatter = source.match(/^---\n([\s\S]*?)\n---/)?.[1];
63 + if (!frontmatter) return [];
64 +
65 + const date = getFrontmatterValue(frontmatter, 'date');
66 + const title = getFrontmatterValue(frontmatter, 'title');
67 + if (!date || !title) return [];
68 +
69 + const url = `/docs/changelog/${date.replace(/-/g, '/')}`;
70 +
71 + return [{
72 + id: `${url}#${title}`,
73 + title,
74 + description: getFrontmatterValue(frontmatter, 'description') ?? '',
75 + url,
76 + structuredData: { headings: [], contents: [] },
77 + keywords: ['changelog'],
78 + } satisfies SearchIndex];
79 + });
80 +}
81 +
82 +type SearchIndex = AdvancedIndex & {
83 + keywords?: string[];
84 +};
85 +
86 +export async function getDocsSearchIndexes(): Promise<SearchIndex[]> {
87 + // Load OpenAPI pages and build full reference source.
88 + const openapiPages = await openapiSource(openapi, {
89 + groupBy: 'tag',
90 + baseDir: 'api-reference',
91 + });
92 +
93 + const fullReferenceSource = loader({
94 + baseUrl: '/reference',
95 + source: multiple({
96 + mdx: reference.toFumadocsSource(),
97 + openapi: openapiPages,
98 + }),
99 + plugins: [lucideIconsPlugin(), openapiPlugin()],
100 + });
101 +
102 + const mdxIndexes = [
103 + ...docsSource.getPages(),
104 + ...cookbooksSource.getPages(),
105 + ...toolkitsSource.getPages(),
106 + ...fullReferenceSource.getPages(),
107 + ].map((page) => ({
108 + id: page.url,
109 + title: page.data.title ?? 'Untitled',
110 + description: page.data.description,
111 + url: page.url,
112 + structuredData: page.data.structuredData,
113 + keywords: 'keywords' in page.data ? (page.data.keywords as string[]) : undefined,
114 + } satisfies SearchIndex));
115 +
116 + return [...mdxIndexes, ...dynamicToolkitIndexes, ...getChangelogIndexes()];
117 +}
118 +
119 +function listContentFiles(dir: string): string[] {
120 + if (!existsSync(dir)) return [];
121 +
122 + return readdirSync(dir, { withFileTypes: true }).flatMap((entry) => {
123 + const path = join(dir, entry.name);
124 + if (entry.isDirectory()) return listContentFiles(path);
125 + if (entry.isFile() && /\.mdx?$/.test(entry.name)) return [path];
126 + return [];
127 + });
128 +}
129 +
130 +function slugifyHeading(value: string): string {
131 + return value
132 + .toLowerCase()
133 + .trim()
134 + .replace(/[<>]/g, '')
135 + .replace(/[`*_~\[\]()]/g, '')
136 + .replace(/&[a-z]+;/g, '')
137 + .replace(/[^\w\s-]/g, '')
138 + .replace(/\s+/g, '-')
139 + .replace(/-+/g, '-');
140 +}
141 +
142 +function stripMdx(value: string): string {
143 + return value
144 + .replace(/^---\n[\s\S]*?\n---\n?/, '')
145 + .replace(/```[\s\S]*?```/g, '')
146 + .replace(/[<>]/g, ' ')
147 + .replace(/\{[^}]*\}/g, ' ')
148 + .replace(/!\[[^\]]*\]\([^)]*\)/g, '')
149 + .replace(/\[([^\]]+)\]\([^)]*\)/g, '$1')
150 + .replace(/[`*_>#-]/g, ' ')
151 + .replace(/\s+/g, ' ')
152 + .trim();
153 +}
154 +
155 +function buildStructuredData(markdown: string) {
156 + const body = markdown.replace(/^---\n[\s\S]*?\n---\n?/, '');
157 + const headings: { content: string; id: string }[] = [];
158 + const contents: { content: string; heading: string | undefined }[] = [];
159 + const lines = body.split('\n');
160 + let currentHeading: string | undefined;
161 + let current: string[] = [];
162 +
163 + function flush() {
164 + const content = stripMdx(current.join('\n'));
165 + if (content) contents.push({ content, heading: currentHeading });
166 + current = [];
167 + }
168 +
169 + for (const line of lines) {
170 + const match = line.match(/^(#{2,6})\s+(.+)$/);
171 + if (match) {
172 + flush();
173 + const content = stripMdx(match[2] ?? '');
174 + if (content) {
175 + const id = slugifyHeading(content);
176 + headings.push({ content, id });
177 + currentHeading = id;
178 + }
179 + continue;
180 + }
181 +
182 + current.push(line);
183 + }
184 +
185 + flush();
186 + if (contents.length === 0) {
187 + const content = stripMdx(body);
188 + if (content) contents.push({ content, heading: undefined });
189 + }
190 +
191 + return { headings, contents };
192 +}
193 +
194 +function urlFromContentPath(path: string): string | undefined {
195 + const rel = relative(join(process.cwd(), 'content'), path).replace(/\\/g, '/');
196 + const withoutExt = rel.replace(/\.mdx?$/, '');
197 + const parts = withoutExt.split('/');
198 + const collection = parts.shift();
199 + if (!collection) return undefined;
200 +
201 + if (collection === 'docs') return `/docs/${parts.join('/')}`.replace(/\/index$/, '');
202 + if (collection === 'cookbooks') return `/cookbooks/${parts.join('/')}`.replace(/\/index$/, '');
203 + if (collection === 'reference') return `/reference/${parts.join('/')}`.replace(/\/index$/, '');
204 + if (collection === 'toolkits') {
205 + if (parts[0] === 'faq') return undefined;
206 + return `/toolkits/${parts.join('/')}`.replace(/\/index$/, '');
207 + }
208 + if (collection === 'changelog') return undefined;
209 +
210 + return undefined;
211 +}
212 +
213 +function getFilesystemIndexes(): SearchIndex[] {
214 + const contentDir = join(process.cwd(), 'content');
215 + return listContentFiles(contentDir).flatMap((file) => {
216 + const url = urlFromContentPath(file);
217 + if (!url) return [];
218 +
219 + const source = readFileSync(file, 'utf8');
220 + const frontmatter = source.match(/^---\n([\s\S]*?)\n---/)?.[1] ?? '';
221 + const title = getFrontmatterValue(frontmatter, 'title');
222 + if (!title) return [];
223 +
224 + return [{
225 + id: url,
226 + title,
227 + description: getFrontmatterValue(frontmatter, 'description'),
228 + url,
229 + structuredData: buildStructuredData(source),
230 + keywords: getFrontmatterValue(frontmatter, 'keywords')?.split(',').map((keyword) => keyword.trim()),
231 + } satisfies SearchIndex];
232 + });
233 +}
234 +
235 +async function getOpenApiSearchIndexes(): Promise<SearchIndex[]> {
236 + const openapiPages = await openapiSource(openapi, {
237 + groupBy: 'tag',
238 + baseDir: 'api-reference',
239 + });
240 +
241 + const openapiOnlySource = loader({
242 + baseUrl: '/reference',
243 + source: openapiPages,
244 + plugins: [lucideIconsPlugin(), openapiPlugin()],
245 + });
246 +
247 + return openapiOnlySource.getPages().map((page) => ({
248 + id: page.url,
249 + title: page.data.title ?? 'Untitled',
250 + description: page.data.description,
251 + url: page.url,
252 + structuredData: page.data.structuredData,
253 + } satisfies SearchIndex));
254 +}
255 +
256 +function toAlgoliaDocument(index: SearchIndex): DocumentRecord {
257 + const searchableKeywords = index.keywords?.filter(Boolean).join(' ');
258 + const description = [index.description, searchableKeywords].filter(Boolean).join('\n');
259 +
260 + return {
261 + _id: index.id,
262 + title: index.title,
263 + description,
264 + breadcrumbs: index.breadcrumbs,
265 + url: index.url,
266 + structured: index.structuredData,
267 + tag: Array.isArray(index.tag) ? index.tag[0] : index.tag,
268 + } satisfies DocumentRecord;
269 +}
270 +
271 +export async function getAlgoliaSearchDocuments(): Promise<DocumentRecord[]> {
272 + const indexes = [
273 + ...getFilesystemIndexes(),
274 + ...dynamicToolkitIndexes,
275 + ...getChangelogIndexes(),
276 + ...await getOpenApiSearchIndexes(),
277 + ];
278 +
279 + const deduped = new Map<string, SearchIndex>();
280 + for (const index of indexes) deduped.set(index.id, index);
281 +
282 + return Array.from(deduped.values()).map(toAlgoliaDocument);
283 +}
docs/package.json
+3
@@ -10,6 +10,7 @@
10 "postinstall": "fumadocs-mdx",
11 "lint": "eslint",
12 "lint:links": "bun scripts/validate-links.ts",
13 + "sync:search": "bun scripts/sync-algolia-search.ts",
14 "generate:toolkits": "bun scripts/generate-toolkits.ts",
15 "generate:meta-tools": "bun scripts/generate-meta-tools.ts",
16 "generate:api-index": "bun scripts/generate-api-index.ts",
@@ -20,6 +21,7 @@
21 "@radix-ui/react-tabs": "^1.1.13",
22 "@shikijs/twoslash": "^3.21.0",
23 "@vercel/analytics": "^1.6.1",
24 + "algoliasearch": "^5",
25 "class-variance-authority": "^0.7.1",
26 "clsx": "^2.1.1",
27 "fumadocs-core": "^16.4.2",
@@ -32,6 +34,7 @@
34 "posthog-js": "^1.319.0",
35 "react": "^19.2.3",
36 "react-dom": "^19.2.3",
37 + "search-insights": "^2",
38 "shiki": "^3.20.0",
39 "tailwind-merge": "^3.4.0"
40 },
docs/scripts/sync-algolia-search.ts new
+40
@@ -0,0 +1,40 @@
1 +import { algoliasearch } from 'algoliasearch';
2 +import { sync } from 'fumadocs-core/search/algolia';
3 +import { getAlgoliaSearchDocuments } from '@/lib/search-index';
4 +
5 +function requireEnv(name: string, fallback?: string): string {
6 + const value = process.env[name] ?? fallback;
7 + if (!value) {
8 + throw new Error(`Missing required environment variable: ${name}`);
9 + }
10 + return value;
11 +}
12 +
13 +const dryRun = process.argv.includes('--dry-run');
14 +const indexName =
15 + process.env.ALGOLIA_INDEX_NAME ?? process.env.NEXT_PUBLIC_ALGOLIA_INDEX_NAME ?? 'docs_composio_dev_62hi9pqz1l_pages';
16 +
17 +const documents = await getAlgoliaSearchDocuments();
18 +
19 +if (documents.length === 0) {
20 + throw new Error('Refusing to sync an empty Algolia docs index.');
21 +}
22 +
23 +if (dryRun) {
24 + console.log(`Prepared ${documents.length} docs pages for Algolia index "${indexName}".`);
25 + process.exit(0);
26 +}
27 +
28 +const appId = requireEnv(
29 + 'ALGOLIA_APP_ID',
30 + process.env.NEXT_PUBLIC_ALGOLIA_APP_ID ?? '62HI9PQZ1L',
31 +);
32 +const adminApiKey = requireEnv('ALGOLIA_ADMIN_API_KEY');
33 +const client = algoliasearch(appId, adminApiKey);
34 +
35 +console.log(`Syncing ${documents.length} docs pages to Algolia index "${indexName}"...`);
36 +await sync(client, {
37 + indexName,
38 + documents,
39 +});
40 +console.log(`Synced Algolia docs search index "${indexName}".`);