@devries-cameron20 / composio-1 / commits / 5350b00d9
Fork 0

Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

perf(cli): resolve toolkits from local knowledge, not the catalog

Every tool execution had to know which toolkit the tool belongs to, and the only reliable source was the toolkit catalog — ~800 KB and ~2 s of the wall time of a command that then does one small API call. Match against what the CLI already knows instead: the slugs baked in at build time, plus the ones this machine has learned since, kept in `known-toolkit-slugs.json`. A slug that matches nothing known still falls through to the catalog — that is what a toolkit released after the binary looks like — and what it finds is remembered, so it costs a fetch once rather than every run. Learned slugs older than a week trigger a background refresh; that bounds how long a brand-new toolkit can be mistaken for a shorter one sharing its prefix. The file is derived data rather than saved API responses, so it is read regardless of FORCE_USE_CACHE, which keeps its meaning. Nothing here can fail a command: a missing, unreadable, or corrupt file means "nothing learned yet", and an unreachable catalog still degrades to the first-underscore guess. Cache writes also move to a temp file and a rename, so an interrupted run leaves the previous cache intact instead of a truncated file that the next run reads back as a parse failure.

jkomyno committed Aug 4, 2026 at 19:31 UTC 5350b00d906c053e8ec366ce3a95a6e4e62e91b2
13 files changed +650 -78
ts/packages/cli/CHANGELOG.md
+16
@@ -44,6 +44,22 @@
44 layer treated the request's own failure as a cache failure and retried it,
45 so every failed toolkit, tool, or trigger listing cost two round trips and
46 twice the wait before reporting the same error.
47 +- Resolving a tool's toolkit no longer downloads the toolkit catalog. The CLI
48 + ships with the toolkit slugs it knew at build time and remembers any it
49 + learns since in `known-toolkit-slugs.json`, so `composio tools execute` only
50 + reaches for the catalog when a slug matches nothing it knows — a toolkit
51 + released after your CLI version. `FORCE_USE_CACHE` is unaffected.
52 +- Resolving a tool's toolkit now reads what the CLI knows locally once per run
53 + rather than once per lookup. A single `composio tools execute` resolves the
54 + same toolkit up to four times — and once per tool with `--parallel` — and
55 + each of those re-read `known-toolkit-slugs.json`, re-parsed it, and rebuilt
56 + the lookup table over it. The weekly background refresh now also runs once per
57 + run instead of rewriting the file after every lookup.
58 +- The background catalog refresh is abandoned after ten seconds. A finished
59 + command ends when the event loop drains, so a slow network could otherwise
60 + hold an exiting `composio` process open until the refresh completed.
61 +- Cache files are now written atomically, so an interrupted run can no longer
62 + leave a truncated `toolkits.json` or `tools.json` behind.
63
64 ## 0.3.1
65
ts/packages/cli/README.md
+9 -1
@@ -129,7 +129,7 @@ The CLI implements a file-based caching system for improved performance and offl
129 ### Cache Features
130
131 - **Cache-first reads**: When `FORCE_USE_CACHE=true`, the CLI first checks for cached data before making API calls. If you already ran `composio generate` before, it will work even if you're offline.
132 -- **Best-effort writes**: All successful API responses are automatically cached to disk for future use.
132 +- **Best-effort writes**: All successful API responses are automatically cached to disk for future use. Writes are atomic — a run interrupted mid-write leaves the previous cache intact rather than a truncated file.
133 - **Graceful fallback**: If cache files are corrupted or missing, the CLI falls back to making API calls.
134 - **Parameter-aware caching**: Methods with parameters include those parameters in the cache key.
135
@@ -148,6 +148,14 @@ The following files are cached:
148 - `trigger-types-as-enums.json` - Results from trigger type enumerations
149 - `trigger-types.json` - Results from paginated trigger types payloads
150
151 +### Known toolkit slugs
152 +
153 +`known-toolkit-slugs.json` also lives in the cache directory, but it is not one of the files above.
154 +
155 +Running a tool means knowing which toolkit it belongs to, and the tool slug alone does not say: `GOOGLE_ANALYTICS_RUN_REPORT` belongs to `google_analytics`, not `google`. The CLI ships with the list of toolkit slugs that existed when it was built and records any it learns afterwards in this file, so resolving a toolkit costs a small local read instead of downloading the catalog.
156 +
157 +It holds derived data, not saved API responses, so it is read on every run regardless of `FORCE_USE_CACHE` — that variable keeps its meaning of opting in to replaying previously cached API responses. Deleting the file is safe: the CLI re-learns what it needs. Being out of date is also safe, because the backend never removes a toolkit; a slug the file has never seen simply costs one catalog fetch, after which it is remembered and refreshed weekly in the background.
158 +
159 ## Development
160
161 ### Installation
ts/packages/cli/src/cli-main.ts
+10 -1
@@ -26,6 +26,7 @@ import { UpgradeBinary } from 'src/services/upgrade-binary';
26 import { TerminalUI, TerminalUILive } from 'src/services/terminal-ui';
27 import { TriggersRealtime } from 'src/services/triggers-realtime';
28 import { ToolsExecutorLive as _ToolsExecutorLive } from 'src/services/tools-executor';
29 +import { ToolkitSlugCatalog } from 'src/services/toolkit-slug-catalog';
30 import { ProjectContext } from 'src/services/project-context';
31 import { ProjectEnvironmentDetector } from 'src/services/project-environment-detector';
32 import { CommandRunner } from 'src/services/command-runner';
@@ -94,9 +95,16 @@ export const ComposioClientSingletonLive = Layer.provide(
95 Layer.mergeAll(BunFileSystem.layer, BunPath.layer, NodeOs.Default, ConfigLive)
96 ) satisfies RequiredLayer;
97
98 +// Fed the cached repository so that the staleness refresh behind it shares the
99 +// one catalog fetch a run is allowed, rather than starting a second.
100 +export const ToolkitSlugCatalogLive = Layer.provide(
101 + ToolkitSlugCatalog.Default,
102 + ComposioToolkitsRepositoryCachedLive
103 +) satisfies RequiredLayer;
104 +
105 export const ToolsExecutorLive = Layer.provide(
106 _ToolsExecutorLive,
99 - ComposioClientSingletonLive
107 + Layer.mergeAll(ComposioClientSingletonLive, ToolkitSlugCatalogLive)
108 ) satisfies RequiredLayer;
109
110 export const ProjectContextLive = Layer.provide(
@@ -119,6 +127,7 @@ const layers = Layer.mergeAll(
127 ComposioSessionRepositoryLive,
128 ComposioClientSingletonLive,
129 ComposioToolkitsRepositoryCachedLive,
130 + ToolkitSlugCatalogLive,
131 ToolsExecutorLive,
132 JsPackageManagerDetector.Default,
133 ProjectEnvironmentDetector.Default,
ts/packages/cli/src/effects/toolkit-from-tool-slug.ts
+28 -11
@@ -1,23 +1,31 @@
1 import { Effect } from 'effect';
2 import { ComposioToolkitsRepository } from 'src/services/composio-clients';
3 +import { ToolkitSlugCatalog } from 'src/services/toolkit-slug-catalog';
4 import { isMetaToolSlug } from 'src/utils/meta-tool-slugs';
5 import {
6 guessToolkitFromToolSlug,
7 matchToolkitFromToolSlug,
8 + toolkitFromMatchedPrefix,
9 } from 'src/utils/toolkit-from-tool-slug';
10
11 /**
10 - * Resolves the toolkit slug for a tool or trigger slug by matching the longest
11 - * known toolkit slug prefix from the toolkit list (served from the local cache
12 - * when warm). No string split can find where the toolkit name ends —
12 + * Resolves the toolkit slug for a tool or trigger slug.
13 + *
14 + * No string split can find where the toolkit name ends —
15 * `GOOGLE_ANALYTICS_RUN_REPORT` belongs to `google_analytics`, not `google` —
14 - * so the known list is the only reliable source. Falls back to the
15 - * first-underscore guess when the list cannot be loaded (first run offline,
16 - * empty cache).
16 + * so this matches the longest known toolkit slug prefix. "Known" is
17 + * {@link ToolkitSlugCatalog}: the catalog baked in at build time plus whatever
18 + * this machine has learned since, both local and memoized per run, so the
19 + * common case costs no network at all.
20 + *
21 + * Only a slug that matches nothing known falls through to the catalog, which
22 + * is what a toolkit released after this binary looks like. Everything else
23 + * degrades rather than fails: an unreachable catalog still yields the
24 + * first-underscore guess.
25 */
26 export const toolkitFromToolSlug = (
27 toolSlug: string
20 -): Effect.Effect<string | undefined, never, ComposioToolkitsRepository> =>
28 +): Effect.Effect<string | undefined, never, ComposioToolkitsRepository | ToolkitSlugCatalog> =>
29 Effect.gen(function* () {
30 // Meta tools belong to the session rather than to a toolkit, and their
31 // slugs shadow real ones — `COMPOSIO_SEARCH_TOOLS` prefix-matches the
@@ -27,10 +35,19 @@ export const toolkitFromToolSlug = (
35 return undefined;
36 }
37
38 + const catalog = yield* ToolkitSlugCatalog;
39 + const local = yield* catalog.local;
40 +
41 + const match = local.match(toolSlug);
42 + if (match !== undefined) {
43 + return toolkitFromMatchedPrefix(match);
44 + }
45 +
46 const repository = yield* ComposioToolkitsRepository;
47 const toolkits = yield* repository.getToolkits();
32 - return matchToolkitFromToolSlug(
33 - toolSlug,
34 - toolkits.map(toolkit => toolkit.slug)
35 - );
48 + const allSlugs = [...local.slugs, ...toolkits.map(toolkit => toolkit.slug)];
49 +
50 + yield* catalog.remember(allSlugs);
51 +
52 + return matchToolkitFromToolSlug(toolSlug, allSlugs);
53 }).pipe(Effect.catchAll(() => Effect.succeed(guessToolkitFromToolSlug(toolSlug))));
ts/packages/cli/src/effects/write-file-atomic.ts new
+26
@@ -0,0 +1,26 @@
1 +import { FileSystem } from '@effect/platform';
2 +import { Effect, Random } from 'effect';
3 +
4 +/**
5 + * Writes `content` to a sibling temp file and renames it over `filePath`.
6 + *
7 + * Renaming within a filesystem is atomic, so a reader — another CLI process,
8 + * or the next run after this one is killed mid-write — sees either the old
9 + * file or the new one, never a truncated mix. The temp file is a sibling
10 + * rather than a tmpdir entry so both paths stay on the same volume, where
11 + * rename cannot degrade into a copy.
12 + */
13 +export const writeFileAtomic = (filePath: string, content: string) =>
14 + Effect.gen(function* () {
15 + const fs = yield* FileSystem.FileSystem;
16 +
17 + // Concurrent writers must not share a temp path, or one would rename the
18 + // other's half-written file into place.
19 + const suffix = yield* Random.nextIntBetween(0x100000, 0xffffff);
20 + const tempPath = `${filePath}.tmp-${suffix.toString(16)}`;
21 +
22 + yield* fs.writeFileString(tempPath, content);
23 + yield* fs
24 + .rename(tempPath, filePath)
25 + .pipe(Effect.onError(() => fs.remove(tempPath).pipe(Effect.ignore)));
26 + });
ts/packages/cli/src/models/tools.ts
-10
@@ -58,16 +58,6 @@ export const Tool = Schema.Struct({
58 * List of tags associated with the tool for categorization and filtering.
59 */
60 tags: Schema.Array(Schema.String),
61 - /**
62 - * Parent toolkit of the tool. Absent in cache entries written before the
63 - * field was captured.
64 - */
65 - toolkit: Schema.optional(
66 - Schema.Struct({
67 - name: Schema.String,
68 - slug: Schema.String,
69 - })
70 - ),
61 }).annotations({ identifier: 'Tool' });
62 export type Tool = Schema.Schema.Type<typeof Tool>;
63
ts/packages/cli/src/services/composio-clients-cached.ts
+9 -3
@@ -3,6 +3,7 @@ import { FileSystem, Path } from '@effect/platform';
3 import { BunFileSystem } from '@effect/platform-bun';
4 import { setupCacheDir } from 'src/effects/setup-cache-dir';
5 import { FORCE_CONFIG } from 'src/effects/force-config';
6 +import { writeFileAtomic } from 'src/effects/write-file-atomic';
7 import { ComposioToolkitsRepository, InvalidToolkitsError } from './composio-clients';
8 import type { ToolkitVersionSpec } from 'src/effects/toolkit-version-overrides';
9 import { NodeOs } from './node-os';
@@ -90,9 +91,14 @@ function createCachedEffect<T, E, R>(
91 )
92 );
93
93 - const writeToCache = (fs: FileSystem.FileSystem, cacheFilePath: string, result: T) =>
94 + /**
95 + * Atomic: these files are hundreds of KB, and a run killed mid-write — or
96 + * two CLI processes writing at once — would otherwise leave behind a
97 + * truncated file that the next run reads back as a parse failure.
98 + */
99 + const writeToCache = (cacheFilePath: string, result: T) =>
100 encoder(result).pipe(
95 - Effect.flatMap(content => fs.writeFileString(cacheFilePath, content)),
101 + Effect.flatMap(content => writeFileAtomic(cacheFilePath, content)),
102 Effect.catchAll(error =>
103 Effect.logWarning(`Failed to write to cache ${cacheFilePath}: ${error}`)
104 )
@@ -130,7 +136,7 @@ function createCachedEffect<T, E, R>(
136 // Write to cache only if we're fetching the full dataset (no cacheFilter).
137 // Filtered API calls fetch partial data that would corrupt the shared cache file.
138 if (!cacheFilter && Option.isSome(cacheFilePath)) {
133 - yield* writeToCache(fs, cacheFilePath.value, result);
139 + yield* writeToCache(cacheFilePath.value, result);
140 }
141
142 return result;
ts/packages/cli/src/services/known-toolkit-slugs.ts new
+96
@@ -0,0 +1,96 @@
1 +import { FileSystem, Path } from '@effect/platform';
2 +import { BunFileSystem } from '@effect/platform-bun';
3 +import { DateTime, Effect, Layer, Option, Schema } from 'effect';
4 +import { setupCacheDir } from 'src/effects/setup-cache-dir';
5 +import { writeFileAtomic } from 'src/effects/write-file-atomic';
6 +import { JSONTransformSchema } from 'src/models/utils/json-transform-schema';
7 +import { NodeOs } from './node-os';
8 +
9 +/**
10 + * Toolkit slugs this machine has learned since the CLI was built.
11 + *
12 + * The CLI ships with the catalog it knew at build time
13 + * (`src/generated/toolkit-slugs.ts`). Toolkits released after that are
14 + * discovered the first time someone runs one of their tools, and remembered
15 + * here so the next run costs nothing.
16 + *
17 + * This is derived data, not an HTTP response cache: it is read unconditionally
18 + * rather than under `FORCE_USE_CACHE`, which stays what it has always been —
19 + * an opt-in to replaying previously cached API responses. Being wrong about
20 + * this file can only cost a fetch, never an answer: an unknown slug falls
21 + * through to the catalog, and the backend never removes a toolkit, so a slug
22 + * recorded here never stops being real.
23 + */
24 +
25 +export const KNOWN_TOOLKIT_SLUGS_FILE = 'known-toolkit-slugs.json';
26 +
27 +const KnownToolkitSlugs = Schema.Struct({
28 + slugs: Schema.Array(Schema.String),
29 + refreshedAt: Schema.DateTimeUtc,
30 +});
31 +
32 +export type KnownToolkitSlugs = Schema.Schema.Type<typeof KnownToolkitSlugs>;
33 +
34 +const KnownToolkitSlugsJSON = JSONTransformSchema(KnownToolkitSlugs);
35 +
36 +/**
37 + * The file lives in the cache directory but is not part of the toolkit cache,
38 + * so it self-provides its platform services: callers stay free of filesystem
39 + * dependencies in their own environments.
40 + */
41 +const provideFileSystem = <A, E>(
42 + effect: Effect.Effect<A, E, FileSystem.FileSystem | Path.Path | NodeOs>
43 +): Effect.Effect<A, E> =>
44 + effect.pipe(Effect.provide(Layer.mergeAll(BunFileSystem.layer, Path.layer, NodeOs.Default)));
45 +
46 +const knownToolkitSlugsPath = Effect.gen(function* () {
47 + const path = yield* Path.Path;
48 + const cacheDir = yield* setupCacheDir;
49 + return path.join(cacheDir, KNOWN_TOOLKIT_SLUGS_FILE);
50 +});
51 +
52 +/**
53 + * Reads the learned slugs. A missing, unreadable, or corrupted file is not a
54 + * failure — it means "nothing learned yet", and the next refresh rewrites it.
55 + */
56 +export const readKnownToolkitSlugs: Effect.Effect<Option.Option<KnownToolkitSlugs>> = Effect.gen(
57 + function* () {
58 + const fs = yield* FileSystem.FileSystem;
59 + const filePath = yield* knownToolkitSlugsPath;
60 + const content = yield* fs.readFileString(filePath);
61 + return yield* Schema.decode(KnownToolkitSlugsJSON)(content);
62 + }
63 +).pipe(
64 + Effect.asSome,
65 + Effect.catchAll(error =>
66 + Effect.logDebug(`No usable ${KNOWN_TOOLKIT_SLUGS_FILE}: ${error}`).pipe(
67 + Effect.as(Option.none<KnownToolkitSlugs>())
68 + )
69 + ),
70 + provideFileSystem
71 +);
72 +
73 +/**
74 + * Records `slugs`, sorted and deduped, stamped with the current time.
75 + *
76 + * Never fails: this is an optimization, and a machine that cannot write to its
77 + * own cache directory should still be able to run tools. Concurrent CLI
78 + * processes are safe by construction — the write is atomic, and since slugs
79 + * are only ever added, whichever writer lands last leaves a usable file.
80 + */
81 +export const writeKnownToolkitSlugs = (slugs: ReadonlyArray<string>): Effect.Effect<void> =>
82 + Effect.gen(function* () {
83 + const filePath = yield* knownToolkitSlugsPath;
84 + const refreshedAt = yield* DateTime.now;
85 +
86 + const content = yield* Schema.encode(KnownToolkitSlugsJSON)({
87 + slugs: [...new Set(slugs.map(slug => slug.toLowerCase()))].sort(),
88 + refreshedAt,
89 + });
90 +
91 + yield* writeFileAtomic(filePath, content);
92 + yield* Effect.logDebug(`Recorded ${slugs.length} toolkit slugs in ${filePath}`);
93 + }).pipe(
94 + Effect.catchAll(error => Effect.logDebug(`Failed to record known toolkit slugs: ${error}`)),
95 + provideFileSystem
96 + );
ts/packages/cli/src/services/toolkit-slug-catalog.ts new
+127
@@ -0,0 +1,127 @@
1 +import { DateTime, Duration, Effect, Option, Ref } from 'effect';
2 +import { BAKED_TOOLKIT_SLUGS } from 'src/generated/toolkit-slugs';
3 +import { ComposioToolkitsRepository } from 'src/services/composio-clients';
4 +import {
5 + readKnownToolkitSlugs,
6 + writeKnownToolkitSlugs,
7 + type KnownToolkitSlugs,
8 +} from 'src/services/known-toolkit-slugs';
9 +import { makeLongestPrefixMatcher } from 'src/utils/toolkit-from-tool-slug';
10 +
11 +/**
12 + * How long learned slugs are trusted before a background refresh is started.
13 + * Only relevant for toolkits released after the last refresh: a slug already
14 + * known is never wrong, so this bounds how long a *new* toolkit can be
15 + * mistaken for a shorter one that shares its prefix (a hypothetical
16 + * `google_analytics_v2` resolving as `google_analytics`).
17 + */
18 +const REFRESH_AFTER = Duration.days(7);
19 +
20 +/**
21 + * A successful run ends by event-loop drain rather than an explicit exit, so a
22 + * daemon fiber mid-fetch holds the finished command open until the network
23 + * answers. This bounds that lingering; an interrupted refresh costs a fetch on
24 + * a later run, nothing more.
25 + */
26 +const REFRESH_TIMEOUT = Duration.seconds(10);
27 +
28 +const learnedSlugs = (learned: Option.Option<KnownToolkitSlugs>): ReadonlyArray<string> =>
29 + Option.match(learned, {
30 + onNone: () => [],
31 + onSome: known => known.slugs,
32 + });
33 +
34 +/**
35 + * Re-reads the catalog and records it, in the background. Failures are
36 + * swallowed: a refresh that does not happen costs a fetch later, nothing more.
37 + */
38 +const refreshKnownToolkitSlugs = Effect.gen(function* () {
39 + const repository = yield* ComposioToolkitsRepository;
40 + const toolkits = yield* repository.getToolkits();
41 + yield* writeKnownToolkitSlugs([...BAKED_TOOLKIT_SLUGS, ...toolkits.map(t => t.slug)]);
42 +}).pipe(Effect.timeout(REFRESH_TIMEOUT), Effect.ignore);
43 +
44 +/**
45 + * Starts a refresh when the learned slugs are missing or older than
46 + * {@link REFRESH_AFTER}. A `refreshedAt` in the future (a clock that jumped)
47 + * counts as fresh — the point is to bound staleness, not to police clocks.
48 + */
49 +const refreshInBackgroundIfStale = (learned: Option.Option<KnownToolkitSlugs>) =>
50 + Effect.gen(function* () {
51 + const now = yield* DateTime.now;
52 + const isFresh = Option.match(learned, {
53 + onNone: () => false,
54 + onSome: known =>
55 + Duration.lessThan(
56 + Duration.millis(DateTime.distance(known.refreshedAt, now)),
57 + REFRESH_AFTER
58 + ),
59 + });
60 +
61 + if (isFresh) return;
62 +
63 + yield* Effect.forkDaemon(refreshKnownToolkitSlugs);
64 + });
65 +
66 +/**
67 + * The toolkit slugs a run can match against without asking the API: the
68 + * catalog baked in at build time plus whatever this machine has learned since.
69 + */
70 +export interface LocalToolkitSlugs {
71 + readonly slugs: ReadonlyArray<string>;
72 + /**
73 + * Longest-prefix match over {@link slugs}. Returns the matched toolkit slug,
74 + * or undefined when nothing local matches — which is what a toolkit released
75 + * after this binary looks like.
76 + */
77 + readonly match: (toolSlug: string) => string | undefined;
78 +}
79 +
80 +const loadLocalToolkitSlugs = Effect.gen(function* () {
81 + const learned = yield* readKnownToolkitSlugs;
82 + const slugs = [...BAKED_TOOLKIT_SLUGS, ...learnedSlugs(learned)];
83 +
84 + yield* refreshInBackgroundIfStale(learned);
85 +
86 + return { slugs, match: makeLongestPrefixMatcher(slugs) } satisfies LocalToolkitSlugs;
87 +});
88 +
89 +/**
90 + * Owns the local half of toolkit resolution, resolved once per process.
91 + *
92 + * Reading it costs a file read, a JSON parse, a schema decode, and a lookup set
93 + * over a thousand-odd slugs — cheap next to the catalog fetch it replaces, but
94 + * a single `composio execute` resolves a toolkit four times over (bootstrap
95 + * telemetry, the execute context, the connection check, the execution itself),
96 + * and `--parallel` multiplies that by the number of tools. None of it can
97 + * change under a running process, so `Effect.cached` memoizes it for the
98 + * lifetime of the layer: the first caller pays, the rest await the same result.
99 + *
100 + * Memoizing also gates the staleness refresh to one fork per run, where an
101 + * unmemoized read forked — and rewrote the file — once per resolution.
102 + */
103 +export class ToolkitSlugCatalog extends Effect.Service<ToolkitSlugCatalog>()(
104 + 'services/ToolkitSlugCatalog',
105 + {
106 + effect: Effect.gen(function* () {
107 + const local = yield* Effect.cached(loadLocalToolkitSlugs);
108 + const hasRecorded = yield* Ref.make(false);
109 +
110 + /**
111 + * Records slugs learned from a catalog fetch, in the background, at most
112 + * once per run. Every miss in a run merges the same memoized fetch into
113 + * the same local list, so later calls would fork a daemon fiber only to
114 + * rewrite an identical file — and each pending fiber holds the finished
115 + * command open a little longer.
116 + */
117 + const remember = (slugs: ReadonlyArray<string>): Effect.Effect<void> =>
118 + Effect.gen(function* () {
119 + const alreadyRecorded = yield* Ref.getAndSet(hasRecorded, true);
120 + if (alreadyRecorded) return;
121 + yield* Effect.forkDaemon(writeKnownToolkitSlugs(slugs));
122 + });
123 +
124 + return { local, remember };
125 + }),
126 + }
127 +) {}
ts/packages/cli/src/services/tools-executor.ts
+8 -1
@@ -16,6 +16,7 @@ import {
16 import { getOrFetchToolInputDefinition } from 'src/services/tool-input-validation';
17 import { ToolFileUploadError, uploadToolInputFiles } from 'src/services/tool-file-uploads';
18 import { toolkitFromToolSlug } from 'src/effects/toolkit-from-tool-slug';
19 +import { ToolkitSlugCatalog } from 'src/services/toolkit-slug-catalog';
20 import { isMetaToolSlug } from 'src/utils/meta-tool-slugs';
21 import type { NodeOs } from 'src/services/node-os';
22 import type { NodeProcess } from 'src/services/node-process';
@@ -131,6 +132,11 @@ export const ToolsExecutorLive = Layer.effect(
132 // the raw Composio client on first call — no environment requirements.
133 const clientSingleton = yield* ComposioClientSingleton;
134
135 + // Held rather than required per call: which slugs resolve locally is an
136 + // implementation detail of toolkit resolution, not part of what a caller
137 + // has to hand the executor.
138 + const slugCatalog = yield* ToolkitSlugCatalog;
139 +
140 return ToolsExecutor.of({
141 execute: (slug, params) =>
142 Effect.gen(function* () {
@@ -252,7 +258,8 @@ export const ToolsExecutorLive = Layer.effect(
258 );
259 })
260 )
255 - )
261 + ),
262 + Effect.provideService(ToolkitSlugCatalog, slugCatalog)
263 ),
264 });
265 })
ts/packages/cli/src/utils/toolkit-from-tool-slug.ts
+30 -10
@@ -24,16 +24,38 @@ export const guessToolkitFromToolSlug = (toolSlug: string): string | undefined =
24 export const longestPrefix = (
25 toolSlug: string,
26 knownToolkitSlugs: ReadonlyArray<string>
27 -): string | undefined => {
27 +): string | undefined => makeLongestPrefixMatcher(knownToolkitSlugs)(toolSlug);
28 +
29 +/**
30 + * {@link longestPrefix} against a list that does not change, with the lookup
31 + * set built once and closed over. The known list runs to a thousand-odd slugs
32 + * and a single command resolves several tool slugs against it, so callers that
33 + * hold onto the returned function pay for the set once instead of per lookup.
34 + */
35 +export const makeLongestPrefixMatcher = (
36 + knownToolkitSlugs: ReadonlyArray<string>
37 +): ((toolSlug: string) => string | undefined) => {
38 const known = new Set(knownToolkitSlugs.map(slug => slug.toLowerCase()));
29 - const segments = toolSlug.toLowerCase().split('_');
30 - for (let end = segments.length - 1; end >= 1; end -= 1) {
31 - const candidate = segments.slice(0, end).join('_');
32 - if (known.has(candidate)) return candidate;
33 - }
34 - return undefined;
39 +
40 + return toolSlug => {
41 + const segments = toolSlug.toLowerCase().split('_');
42 + for (let end = segments.length - 1; end >= 1; end -= 1) {
43 + const candidate = segments.slice(0, end).join('_');
44 + if (known.has(candidate)) return candidate;
45 + }
46 + return undefined;
47 + };
48 };
49
50 +/**
51 + * Bare `composio`-prefixed slugs are meta tools, not user-linkable toolkits;
52 + * longer matches such as `composio_search` are real toolkits. A `composio`
53 + * match is a resolved answer rather than a miss, so callers must not fall
54 + * through to a wider search on the `undefined` it returns.
55 + */
56 +export const toolkitFromMatchedPrefix = (match: string): string | undefined =>
57 + match === 'composio' ? undefined : match;
58 +
59 /**
60 * `longestPrefix` with the toolkit-resolution policy applied on top: the
61 * `composio` meta guard, and `guessToolkitFromToolSlug` when no candidate is
@@ -45,9 +67,7 @@ export const matchToolkitFromToolSlug = (
67 ): string | undefined => {
68 const match = longestPrefix(toolSlug, knownToolkitSlugs);
69 if (match !== undefined) {
48 - // Bare `composio`-prefixed slugs are meta tools, not user-linkable
49 - // toolkits; longer matches such as `composio_search` are real toolkits.
50 - return match === 'composio' ? undefined : match;
70 + return toolkitFromMatchedPrefix(match);
71 }
72 return guessToolkitFromToolSlug(toolSlug);
73 };
ts/packages/cli/test/__utils__/services/test-layer.ts
+13 -1
@@ -47,6 +47,7 @@ import { UpgradeBinary } from 'src/services/upgrade-binary';
47 import { NodeOs } from 'src/services/node-os';
48 import { TriggersRealtime } from 'src/services/triggers-realtime';
49 import { ToolsExecutor, ToolsExecutorLive } from 'src/services/tools-executor';
50 +import { ToolkitSlugCatalog } from 'src/services/toolkit-slug-catalog';
51 import type { ToolExecuteResponse } from 'src/services/tools-executor';
52 import type {
53 SessionCreateResponse,
@@ -1214,6 +1215,13 @@ export const TestLayer = (input?: TestLiveInput) =>
1215 })
1216 );
1217
1218 + // Built per test layer, so each test resolves toolkit slugs against its own
1219 + // cache directory rather than inheriting a memo from an earlier test.
1220 + const ToolkitSlugCatalogTest = Layer.provide(
1221 + ToolkitSlugCatalog.Default,
1222 + ComposioToolkitsRepositoryTest
1223 + );
1224 +
1225 // --- ToolsExecutor ---
1226 // When `input.toolsExecutor` is set, use a canned mock (bypasses Tool Router).
1227 // Otherwise, use the real ToolsExecutorLive which flows through the mock ComposioClientSingleton.
@@ -1237,7 +1245,10 @@ export const TestLayer = (input?: TestLiveInput) =>
1245 },
1246 })
1247 )
1240 - : Layer.provide(ToolsExecutorLive, ComposioClientSingletonTest);
1248 + : Layer.provide(
1249 + ToolsExecutorLive,
1250 + Layer.mergeAll(ComposioClientSingletonTest, ToolkitSlugCatalogTest)
1251 + );
1252
1253 const CliConfigLive = CliConfig.layer(ComposioCliConfig);
1254
@@ -1286,6 +1297,7 @@ export const TestLayer = (input?: TestLiveInput) =>
1297 ComposioSessionRepositoryTest,
1298 TriggersRealtimeTest,
1299 ComposioToolkitsRepositoryTest,
1300 + ToolkitSlugCatalogTest,
1301 JsPackageManagerDetector.Default,
1302 ProjectEnvironmentDetector.Default,
1303 CommandRunnerTest,
ts/packages/cli/test/src/effects/toolkit-from-tool-slug.test.ts
+278 -40
@@ -1,63 +1,301 @@
1 -import { describe, expect, layer } from '@effect/vitest';
2 -import { Effect } from 'effect';
1 +import { describe, expect, it } from '@effect/vitest';
2 +import { FileSystem } from '@effect/platform';
3 +import { BunFileSystem } from '@effect/platform-bun';
4 +import { ConfigProvider, DateTime, Effect, Layer, Schedule } from 'effect';
5 +import * as tempy from 'tempy';
6 import { toolkitFromToolSlug } from 'src/effects/toolkit-from-tool-slug';
7 import type { Toolkits } from 'src/models/toolkits';
5 -import { TestLive } from 'test/__utils__';
8 +import { ComposioToolkitsRepository, HttpServerError } from 'src/services/composio-clients';
9 +import { KNOWN_TOOLKIT_SLUGS_FILE } from 'src/services/known-toolkit-slugs';
10 +import { ToolkitSlugCatalog } from 'src/services/toolkit-slug-catalog';
11 import { makeToolkitFixture } from 'test/__utils__/models/toolkits';
12 +import {
13 + countingToolkitsRepository,
14 + type GetToolkitsError,
15 +} from 'test/__utils__/services/toolkits-repository-stub';
16
8 -const testToolkits: Toolkits = [
9 - makeToolkitFixture('google', 'Google'),
10 - makeToolkitFixture('google_analytics', 'Google Analytics'),
11 - makeToolkitFixture('gmail', 'Gmail'),
12 - makeToolkitFixture('microsoft', 'Microsoft'),
13 - makeToolkitFixture('microsoft_teams', 'Microsoft Teams'),
14 - // A real, linkable toolkit whose slug is also the prefix of a session meta
15 - // tool (`COMPOSIO_SEARCH_TOOLS`).
16 - makeToolkitFixture('composio_search', 'Composio Search'),
17 -];
17 +/**
18 + * A toolkit that cannot be in the baked catalog, standing in for one released
19 + * after the binary was built. Tests of the fetch path must use slugs like this
20 + * — anything real resolves locally and never reaches the repository.
21 + */
22 +const UNRELEASED_TOOLKIT = 'acme_analytics';
23
19 -describe('toolkitFromToolSlug', () => {
20 - layer(TestLive({ toolkitsData: { toolkits: testToolkits } }))('with known toolkits', it => {
21 - it.effect('resolves multi-word toolkit slugs by longest known prefix', () =>
22 - Effect.gen(function* () {
23 - expect(yield* toolkitFromToolSlug('GOOGLE_ANALYTICS_RUN_REPORT')).toBe('google_analytics');
24 - expect(yield* toolkitFromToolSlug('MICROSOFT_TEAMS_SEND_MESSAGE')).toBe('microsoft_teams');
25 - })
24 +const failingFetch = () =>
25 + Effect.fail(new HttpServerError({ cause: 'catalog unavailable', status: 503 }));
26 +
27 +/**
28 + * A learned-slugs file recorded `daysAgo` days ago. Anything under a week old
29 + * keeps the resolver from starting a background refresh, which is what makes
30 + * "no catalog fetch" assertions meaningful.
31 + */
32 +const learnedFileContent = (slugs: ReadonlyArray<string>, daysAgo = 0) =>
33 + JSON.stringify({
34 + slugs,
35 + refreshedAt: DateTime.formatIso(DateTime.subtract(DateTime.unsafeNow(), { days: daysAgo })),
36 + });
37 +
38 +interface ResolverContext {
39 + readonly calls: () => number;
40 + readonly waitForLearnedFile: (
41 + predicate: (content: string) => boolean
42 + ) => Effect.Effect<string, unknown, FileSystem.FileSystem>;
43 +}
44 +
45 +/**
46 + * Runs `program` against a private cache directory, so the resolver reads and
47 + * writes a learned-slugs file belonging to this test alone. `seedLearnedFile`
48 + * is written verbatim — including deliberately corrupt content.
49 + */
50 +const withResolver = <A>(
51 + options: {
52 + readonly getToolkits?: () => Effect.Effect<Toolkits, GetToolkitsError>;
53 + readonly seedLearnedFile?: string;
54 + },
55 + program: (
56 + context: ResolverContext
57 + ) => Effect.Effect<
58 + A,
59 + unknown,
60 + FileSystem.FileSystem | ComposioToolkitsRepository | ToolkitSlugCatalog
61 + >
62 +) =>
63 + Effect.suspend(() => {
64 + const cacheDir = tempy.temporaryDirectory();
65 + return runInCacheDir(cacheDir, options, program);
66 + });
67 +
68 +const runInCacheDir = <A>(
69 + cacheDir: string,
70 + options: {
71 + readonly getToolkits?: () => Effect.Effect<Toolkits, GetToolkitsError>;
72 + readonly seedLearnedFile?: string;
73 + },
74 + program: (
75 + context: ResolverContext
76 + ) => Effect.Effect<
77 + A,
78 + unknown,
79 + FileSystem.FileSystem | ComposioToolkitsRepository | ToolkitSlugCatalog
80 + >
81 +) =>
82 + Effect.gen(function* () {
83 + const fs = yield* FileSystem.FileSystem;
84 + const learnedFile = `${cacheDir}/${KNOWN_TOOLKIT_SLUGS_FILE}`;
85 +
86 + if (options.seedLearnedFile !== undefined) {
87 + yield* fs.writeFileString(learnedFile, options.seedLearnedFile);
88 + }
89 +
90 + const repository = countingToolkitsRepository(
91 + options.getToolkits ?? (() => Effect.succeed([] as Toolkits))
92 );
93
28 - it.effect('resolves single-word toolkit slugs', () =>
29 - Effect.gen(function* () {
30 - expect(yield* toolkitFromToolSlug('GMAIL_SEND_EMAIL')).toBe('gmail');
31 - })
94 + const readLearnedFile = fs.readFileString(learnedFile);
95 +
96 + return yield* program({
97 + calls: repository.calls,
98 + // The resolver records what it learns in the background, so a test that
99 + // asserts on the file has to wait for it rather than read once.
100 + waitForLearnedFile: predicate =>
101 + readLearnedFile.pipe(
102 + Effect.filterOrFail(predicate, () => 'learned file has not caught up yet' as const),
103 + Effect.retry(Schedule.spaced('10 millis').pipe(Schedule.intersect(Schedule.recurs(200))))
104 + ),
105 + }).pipe(
106 + // Built inside this test's cache directory, and fresh per test: the
107 + // catalog memoizes what it reads, so a shared one would leak the first
108 + // test's learned slugs into every later one.
109 + Effect.provide(
110 + Layer.mergeAll(
111 + repository.layer,
112 + Layer.provide(ToolkitSlugCatalog.Default, repository.layer)
113 + )
114 + )
115 );
116 + }).pipe(
117 + Effect.provide(BunFileSystem.layer),
118 + Effect.withConfigProvider(ConfigProvider.fromMap(new Map([['CACHE_DIR', cacheDir]])))
119 + );
120
34 - it.effect('falls back to the guess for slugs outside the known list', () =>
121 +describe('toolkitFromToolSlug', () => {
122 + it.effect('resolves a baked multi-word toolkit without touching the network', () =>
123 + withResolver({ seedLearnedFile: learnedFileContent([]) }, ({ calls }) =>
124 Effect.gen(function* () {
36 - expect(yield* toolkitFromToolSlug('NOTION_CREATE_PAGE')).toBe('notion');
125 + expect(yield* toolkitFromToolSlug('GOOGLE_ANALYTICS_RUN_REPORT')).toBe('google_analytics');
126 + expect(yield* toolkitFromToolSlug('GMAIL_SEND_EMAIL')).toBe('gmail');
127 + expect(calls()).toBe(0);
128 })
38 - );
129 + )
130 + );
131
40 - it.effect('gives meta tools no toolkit, even when one shadows their slug', () =>
132 + it.effect('gives meta tools no toolkit, even when one shadows their slug', () =>
133 + withResolver({ seedLearnedFile: learnedFileContent([]) }, ({ calls }) =>
134 Effect.gen(function* () {
42 - // Attributing this to `composio_search` would make a failed meta call
43 - // tell the user to link an app that has nothing to do with it.
135 + // `composio_search` is a real, linkable toolkit whose slug is a prefix
136 + // of this session meta tool. Attributing the meta tool to it would
137 + // send users to link an app they do not need.
138 expect(yield* toolkitFromToolSlug('COMPOSIO_SEARCH_TOOLS')).toBeUndefined();
139 expect(yield* toolkitFromToolSlug('COMPOSIO_MANAGE_CONNECTIONS')).toBeUndefined();
140 + expect(calls()).toBe(0);
141 })
47 - );
142 + )
143 + );
144
49 - it.effect('still resolves ordinary tools of the shadowed toolkit', () =>
145 + it.effect('treats a bare `composio` match as resolved, not as a miss', () =>
146 + withResolver({ seedLearnedFile: learnedFileContent([]) }, ({ calls }) =>
147 Effect.gen(function* () {
51 - expect(yield* toolkitFromToolSlug('COMPOSIO_SEARCH_SEARCH')).toBe('composio_search');
148 + expect(yield* toolkitFromToolSlug('COMPOSIO_ENABLE_TRIGGER')).toBeUndefined();
149 + // A miss here would cost every `COMPOSIO_*` tool a catalog fetch.
150 + expect(calls()).toBe(0);
151 })
53 - );
54 - });
152 + )
153 + );
154 +
155 + it.live('answers from the baked list on a first run, and seeds the file behind it', () =>
156 + withResolver(
157 + { getToolkits: () => Effect.succeed([makeToolkitFixture(UNRELEASED_TOOLKIT)]) },
158 + ({ waitForLearnedFile }) =>
159 + Effect.gen(function* () {
160 + expect(yield* toolkitFromToolSlug('GOOGLE_ANALYTICS_RUN_REPORT')).toBe(
161 + 'google_analytics'
162 + );
163 +
164 + const learned = yield* waitForLearnedFile(content =>
165 + content.includes(UNRELEASED_TOOLKIT)
166 + );
167 + expect(learned).toContain('google_analytics');
168 + })
169 + )
170 + );
171 +
172 + it.live('resolves a learned toolkit without touching the network', () =>
173 + withResolver(
174 + {
175 + seedLearnedFile: learnedFileContent([UNRELEASED_TOOLKIT]),
176 + },
177 + ({ calls }) =>
178 + Effect.gen(function* () {
179 + expect(yield* toolkitFromToolSlug('ACME_ANALYTICS_RUN_REPORT')).toBe(UNRELEASED_TOOLKIT);
180 + expect(calls()).toBe(0);
181 + })
182 + )
183 + );
184
56 - layer(TestLive())('with an empty toolkit list', it => {
57 - it.effect('falls back to the first-underscore guess', () =>
185 + it.live('fetches the catalog for an unknown slug, and remembers what it finds', () =>
186 + withResolver(
187 + {
188 + getToolkits: () => Effect.succeed([makeToolkitFixture(UNRELEASED_TOOLKIT)]),
189 + seedLearnedFile: learnedFileContent([]),
190 + },
191 + ({ calls, waitForLearnedFile }) =>
192 + Effect.gen(function* () {
193 + expect(yield* toolkitFromToolSlug('ACME_ANALYTICS_RUN_REPORT')).toBe(UNRELEASED_TOOLKIT);
194 + expect(calls()).toBe(1);
195 +
196 + const learned = yield* waitForLearnedFile(content =>
197 + content.includes(UNRELEASED_TOOLKIT)
198 + );
199 + // Baked slugs are recorded alongside it, so the file stands alone.
200 + expect(learned).toContain('google_analytics');
201 + })
202 + )
203 + );
204 +
205 + it.live('records learned slugs at most once per run', () =>
206 + withResolver({ seedLearnedFile: learnedFileContent([]) }, ({ waitForLearnedFile }) =>
207 Effect.gen(function* () {
59 - expect(yield* toolkitFromToolSlug('GOOGLE_ANALYTICS_RUN_REPORT')).toBe('google');
208 + const catalog = yield* ToolkitSlugCatalog;
209 + // Every miss in a run merges the same memoized fetch, so a second
210 + // recording could only rewrite the same file. Passing a different list
211 + // here makes the gate observable: its slugs must never land.
212 + yield* catalog.remember([UNRELEASED_TOOLKIT]);
213 + yield* catalog.remember(['slug_from_a_second_recording']);
214 +
215 + const learned = yield* waitForLearnedFile(content => content.includes(UNRELEASED_TOOLKIT));
216 + expect(learned).not.toContain('slug_from_a_second_recording');
217 })
61 - );
62 - });
218 + )
219 + );
220 +
221 + it.live('falls back to the first-underscore guess when the catalog is unreachable', () =>
222 + withResolver(
223 + {
224 + getToolkits: failingFetch,
225 + seedLearnedFile: learnedFileContent([]),
226 + },
227 + ({ calls }) =>
228 + Effect.gen(function* () {
229 + expect(yield* toolkitFromToolSlug('ACME_ANALYTICS_RUN_REPORT')).toBe('acme');
230 + expect(calls()).toBe(1);
231 + })
232 + )
233 + );
234 +
235 + it.live('tolerates a corrupted learned-slugs file', () =>
236 + withResolver({ seedLearnedFile: '{ not json' }, ({ calls }) =>
237 + Effect.gen(function* () {
238 + expect(yield* toolkitFromToolSlug('GOOGLE_ANALYTICS_RUN_REPORT')).toBe('google_analytics');
239 + expect(calls()).toBeLessThanOrEqual(1);
240 + })
241 + )
242 + );
243 +
244 + it.live('refreshes learned slugs in the background once they go stale', () =>
245 + withResolver(
246 + {
247 + getToolkits: () => Effect.succeed([makeToolkitFixture(UNRELEASED_TOOLKIT)]),
248 + seedLearnedFile: learnedFileContent(['stale_toolkit'], 8),
249 + },
250 + ({ waitForLearnedFile }) =>
251 + Effect.gen(function* () {
252 + // Answered from the baked list — the refresh is what the fetch is for.
253 + expect(yield* toolkitFromToolSlug('GOOGLE_ANALYTICS_RUN_REPORT')).toBe(
254 + 'google_analytics'
255 + );
256 +
257 + const learned = yield* waitForLearnedFile(content =>
258 + content.includes(UNRELEASED_TOOLKIT)
259 + );
260 + expect(learned).not.toContain('stale_toolkit');
261 + })
262 + )
263 + );
264 +
265 + it.live('reads local knowledge once, however many slugs a run resolves', () =>
266 + withResolver(
267 + {
268 + getToolkits: () => Effect.succeed([makeToolkitFixture(UNRELEASED_TOOLKIT)]),
269 + seedLearnedFile: learnedFileContent(['stale_toolkit'], 8),
270 + },
271 + ({ calls, waitForLearnedFile }) =>
272 + Effect.gen(function* () {
273 + expect(yield* toolkitFromToolSlug('GOOGLE_ANALYTICS_RUN_REPORT')).toBe(
274 + 'google_analytics'
275 + );
276 + expect(yield* toolkitFromToolSlug('GMAIL_SEND_EMAIL')).toBe('gmail');
277 + expect(yield* toolkitFromToolSlug('GITHUB_CREATE_AN_ISSUE')).toBe('github');
278 +
279 + yield* waitForLearnedFile(content => content.includes(UNRELEASED_TOOLKIT));
280 + // The staleness verdict cannot change under a running process, so the
281 + // refresh it triggers belongs to the run, not to each resolution.
282 + expect(calls()).toBe(1);
283 + })
284 + )
285 + );
286 +
287 + it.live('does not refresh while the learned slugs are fresh', () =>
288 + withResolver(
289 + {
290 + getToolkits: failingFetch,
291 + seedLearnedFile: learnedFileContent([UNRELEASED_TOOLKIT], 6),
292 + },
293 + ({ calls }) =>
294 + Effect.gen(function* () {
295 + expect(yield* toolkitFromToolSlug('ACME_ANALYTICS_RUN_REPORT')).toBe(UNRELEASED_TOOLKIT);
296 + yield* Effect.sleep('50 millis');
297 + expect(calls()).toBe(0);
298 + })
299 + )
300 + );
301 });