@devries-cameron20 / composio-1 / commits / 0a1464d5e
Fork 0

Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

perf(cli): move the compiler and tokenizer out of the executable

`composio --version` goes from 288ms to 199ms, peak RSS from 97.8MB to 77.3MB, and the executable from 85.9MB to 79.7MB. Every command benefits. A compiled Bun binary parses its whole embedded bundle before the first line of JavaScript runs, and #4468 had already made sure the TypeScript compiler and the tokenizer rank table were never *evaluated* unless `generate`, `run`, or a large `execute` response needed them. They were still *parsed* on every start: the compiler alone was 44% of the executable's JavaScript and the o200k rank table another 28%, so `--version` spent ~75ms reading code it could never call. Both now ship as companion modules next to the executable, through the mechanism `composio run` already uses for its own runtime helpers: - `generation-runtime.mjs` carries `src/generation/*`, the `composio run` source rewrites, `typescript`, `@composio/ts-builders` and `openapi-typescript`. `generate ts`, `generate py` and `run` load it with `loadInstalledCompanionModule`; from a source checkout the loader resolves the `.ts` next to `run-companion-modules.ts` instead, so tests and `bun run src/bin.ts` need no build step. - `execute-output-encoder-runtime.mjs` carries `js-tiktoken/lite` and the rank table. `execute` loads it only once a response exceeds the 10KB byte pre-filter. A companion bundles its own copy of `effect`, and a fiber cannot run primitives built by another copy of the runtime, so nothing Effect-shaped crosses the boundary: the generation companion exposes plain functions and promises, runs its pipelines on its own runtime, and returns failures as values that `src/generation/errors.ts` rebuilds as the CLI's own error classes, stack included. Generated output is byte-identical to #4468 for `generate ts`, `generate ts --transpiled` and `generate py`. Both modules join `RUN_COMPANION_MODULE_BASENAMES`, so the build, release packaging, install verification, `upgrade` and the self-repair download pick them up unchanged. The three hand-maintained uninstall lists and the upgrade E2E fixture gain the two file names. Two smaller startup costs go with it: - `src/constants.ts` imported `constants` from `@composio/core`'s root entry for two strings and two URLs, which evaluated the whole SDK at startup (~25ms of module-scope work, mostly zod schemas). The four values are spelled out and pinned to core's by a test. - `tool-file-uploads.ts` imported three core helpers at module scope that only a file upload reaches; they are imported on that path now. The binary build gains a guard: after bundling the companions it bundles `src/bin.ts` once more unminified and fails if the executable's graph reaches `typescript`, `js-tiktoken`, core's root entry, `src/generation/*` or a companion entry. Without it a stray static import would put the compiler back into the executable with nothing to notice. Building also surfaced that `assertBundledRuntimeFiles` blanked string literals to same-length runs of spaces, which made the import patterns' `^\s*` backtrack quadratically across the compiler's multi-megabyte embedded lib strings and stalled the build for over ten minutes. String bodies are dropped now. (The check itself has never matched a specifier, since the specifiers it looks for are the string literals it removes; that is left as it was.) Measured on the pinned toolchain, Bun 1.4.1+4661e494f, linux-x64, best of 15, telemetry disabled, both binaries built in the same session: composio --version 288ms -> 199ms tools execute --help 287ms -> 202ms peak RSS 97.8MB -> 77.3MB executable 85.9MB -> 79.7MB executable JavaScript 8.3MB -> 2.1MB (minified) The `execute` tail after `execute.tool_call.end` is unchanged for responses under 10KB (~10ms) and ~20ms slower above it (351 -> 374ms), which is the on-demand parse of the 2.2MB encoder companion. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wx9gEjuiHux2weiHjdNcDs

DakshM on Exe (exe.dev) committed Sep 13, 2026 at 10:34 UTC 0a1464d5e851f248095ed7fe02373f6decc97300
23 files changed +610 -126
.github/workflows/cli.install-health-check.yml
+3 -1
@@ -90,7 +90,9 @@ jobs:
90 "$HOME/.composio/run-subagent-shared.mjs" \
91 "$HOME/.composio/run-subagent-acp.mjs" \
92 "$HOME/.composio/run-subagent-legacy.mjs" \
93 - "$HOME/.composio/run-subagent-output-mcp.mjs"
93 + "$HOME/.composio/run-subagent-output-mcp.mjs" \
94 + "$HOME/.composio/generation-runtime.mjs" \
95 + "$HOME/.composio/execute-output-encoder-runtime.mjs"
96 rm -rf \
97 "$HOME/.composio/services" \
98 "$HOME/.composio/acp-adapters" \
INSTALL.md
+3 -1
@@ -101,7 +101,9 @@ rm -f \
101 "$install_dir/run-subagent-shared.mjs" \
102 "$install_dir/run-subagent-acp.mjs" \
103 "$install_dir/run-subagent-legacy.mjs" \
104 - "$install_dir/run-subagent-output-mcp.mjs"
104 + "$install_dir/run-subagent-output-mcp.mjs" \
105 + "$install_dir/generation-runtime.mjs" \
106 + "$install_dir/execute-output-encoder-runtime.mjs"
107 rm -rf \
108 "$install_dir/services" \
109 "$install_dir/acp-adapters" \
docs/content/docs/cli.mdx
+3 -1
@@ -171,7 +171,9 @@ rm -f \
171 "$install_dir/run-subagent-shared.mjs" \
172 "$install_dir/run-subagent-acp.mjs" \
173 "$install_dir/run-subagent-legacy.mjs" \
174 - "$install_dir/run-subagent-output-mcp.mjs"
174 + "$install_dir/run-subagent-output-mcp.mjs" \
175 + "$install_dir/generation-runtime.mjs" \
176 + "$install_dir/execute-output-encoder-runtime.mjs"
177 rm -rf \
178 "$install_dir/services" \
179 "$install_dir/acp-adapters" \
ts/e2e-tests/cli/upgrade/e2e.test.ts
+2
@@ -15,6 +15,8 @@ const companionRelativePaths = [
15 'run-subagent-acp.mjs',
16 'run-subagent-legacy.mjs',
17 'run-subagent-output-mcp.mjs',
18 + 'generation-runtime.mjs',
19 + 'execute-output-encoder-runtime.mjs',
20 // RUN_COMPANION_ALL_STATIC_ASSET_RELATIVE_PATHS
21 'acp-adapters/claude-code-acp.mjs',
22 'acp-adapters/cli.js',
ts/packages/cli/AGENTS.md
+2
@@ -97,6 +97,8 @@ Pipeline for `composio generate {ts,py}`:
97
98 `--type-tools` includes full type definitions.
99
100 +Steps 3–4 (and the TypeScript compiler they need) ship as the `generation-runtime` companion module — `dist/generation-runtime.mjs` next to the executable, not inside it — and the tokenizer behind `composio execute`'s inline-vs-file decision ships the same way as `execute-output-encoder-runtime`. Commands reach them through `loadInstalledCompanionModule` (`src/services/run-companion-modules.ts`), never by importing `src/generation/*`, `src/commands/run-source-transforms.ts`, `typescript`, or `js-tiktoken` directly: the binary build fails if the executable's bundle reaches any of those (`assertExecutableExcludesCompanionModules` in `scripts/_shared.ts`). A companion bundles its own copy of `effect`, so its API is plain functions and promises; failures cross as values that `src/generation/errors.ts` turns back into the CLI's error classes.
101 +
102 ### Configuration
103
104 - CLI: `cli-config.ts` defines `ComposioCliConfig` (`builtIns: [GlobalFlag.Help]`, `effect/unstable/cli`'s `CliConfig.Service` shrank to just that one field in v4) — the only `CliConfig` customization Composio makes, wired into `cli-main.ts`'s layer stack as `CliConfigLive`. v3's `autoCorrectLimit`/`isCaseSensitive` have no v4 config equivalent, and neither is reproduced anymore: "Did you mean?" suggestions on `UnrecognizedOption`/`UnknownSubcommand` now render as v4's parser always computes them (no config knob exists to disable them, and Composio wants them), and v4's parser performs no case-folding at all, so case-sensitivity needs no knob. `cli-main.ts` provides no custom `CliOutput.Formatter` — `Command.runWith` uses v4's own `CliOutput.defaultFormatter()`. `GlobalFlag.Version` is deliberately not enabled: `composio --version`, `composio -v`, and the `composio version` command all print the same bare `pkg.version` via `ui.output()`, because `src/commands/index.ts` rewrites the flag spellings to the `version` command before parsing (`normalizeVersionFlag`).
ts/packages/cli/scripts/_shared.ts
+84 -6
@@ -124,19 +124,21 @@ const stripStringsAndComments = (source: string): string => {
124
125 if (char === "'" || char === '"' || char === '`') {
126 const quote = char;
127 - appendSpace(char);
127 + // The literal's body is dropped rather than blanked out. Blanking kept a
128 + // same-length run of spaces, and once a bundle carries a multi-megabyte
129 + // string with newlines in it (the TypeScript compiler's embedded lib
130 + // files, in `generation-runtime`), the `^\s*` prefix of the import
131 + // patterns below backtracks across that run from every line start —
132 + // quadratic, and it stalled the build for over ten minutes.
133 + result += ' ';
134 index += 1;
135
136 while (index < source.length) {
137 const current = source[index];
132 - appendSpace(current);
138 index += 1;
139
140 if (current === '\\') {
136 - if (index < source.length) {
137 - appendSpace(source[index]!);
138 - index += 1;
139 - }
141 + index += 1;
142 continue;
143 }
144
@@ -145,6 +147,7 @@ const stripStringsAndComments = (source: string): string => {
147 }
148 }
149
150 + result += ' ';
151 continue;
152 }
153
@@ -320,6 +323,80 @@ const buildCompanionServiceBundles = async (outputDir: string): Promise<void> =>
323 }
324 };
325
326 +// Module paths that must only ever be reached through a companion module. The
327 +// executable's bundle is checked after every build because the guard against
328 +// regressions is otherwise invisible: a stray static import of any of these
329 +// would silently put the TypeScript compiler or the tokenizer rank table back
330 +// into the executable, and `--version` would quietly get ~70ms slower.
331 +const EXECUTABLE_EXCLUDED_MODULE_PATTERNS: ReadonlyArray<{
332 + readonly pattern: RegExp;
333 + readonly reason: string;
334 +}> = [
335 + { pattern: /\/node_modules\/typescript\//, reason: 'the TypeScript compiler' },
336 + { pattern: /\/node_modules\/js-tiktoken\//, reason: 'the tokenizer and its rank table' },
337 + { pattern: /\/packages\/core\/dist\/index\.mjs$/, reason: "@composio/core's root entry" },
338 + { pattern: /(?:^|\/)src\/generation\/(?!errors\.ts$)/, reason: 'the generation pipeline' },
339 + {
340 + pattern: /(?:^|\/)src\/commands\/run-source-transforms\.ts$/,
341 + reason: 'the run source rewrites',
342 + },
343 + { pattern: /(?:^|\/)src\/services\/generation-runtime\.ts$/, reason: 'the generation companion' },
344 + {
345 + pattern: /(?:^|\/)src\/services\/execute-output-encoder-runtime\.ts$/,
346 + reason: 'the encoder companion',
347 + },
348 +];
349 +
350 +// Bun's unminified output starts each module with a `// <path>` line, which is
351 +// the only bundle-level record of what made it into the graph. A dynamic
352 +// `import()` of a *literal* specifier is still bundled (lazily evaluated, but
353 +// parsed on every start), so this catches those too — only the runtime-computed
354 +// specifier in `loadInstalledCompanionModule` keeps a module out.
355 +const assertExecutableExcludesCompanionModules = async (): Promise<void> => {
356 + const result = await Bun.build({
357 + entrypoints: [path.resolve('./src/bin.ts')],
358 + target: 'bun',
359 + format: 'esm',
360 + packages: 'bundle',
361 + minify: false,
362 + sourcemap: 'none',
363 + });
364 + if (!result.success) {
365 + throw new Error(
366 + `Failed to bundle src/bin.ts for the executable graph check:\n${result.logs
367 + .map(log => log.message)
368 + .join('\n')}`
369 + );
370 + }
371 +
372 + const violations: string[] = [];
373 + for (const artifact of result.outputs) {
374 + const source = await artifact.text();
375 + for (const line of source.split('\n')) {
376 + if (!line.startsWith('// ') || line.startsWith('// @bun')) {
377 + continue;
378 + }
379 + const modulePath = line.slice(3);
380 + const hit = EXECUTABLE_EXCLUDED_MODULE_PATTERNS.find(({ pattern }) =>
381 + pattern.test(modulePath)
382 + );
383 + if (hit) {
384 + violations.push(`${modulePath} (${hit.reason})`);
385 + }
386 + }
387 + }
388 +
389 + if (violations.length > 0) {
390 + throw new Error(
391 + [
392 + 'The executable bundle reaches modules that must only be loaded through a companion module',
393 + '(see RUN_COMPANION_MODULE_BASENAMES). Import them via `loadInstalledCompanionModule` instead:',
394 + ...violations.map(violation => ` - ${violation}`),
395 + ].join('\n')
396 + );
397 + }
398 +};
399 +
400 export const buildCompanionModules = (
401 outputDir: string,
402 options: {
@@ -341,4 +418,5 @@ export const buildCompanionModules = (
418 copyBundledAcpAdapters(outputDir, options.codexBinaryTargets ?? RUN_CODEX_ACP_BINARY_TARGETS)
419 );
420 yield* Effect.tryPromise(() => assertBundledRuntimeFiles(outputDir));
421 + yield* Effect.tryPromise(() => assertExecutableExcludesCompanionModules());
422 });
ts/packages/cli/src/commands/py/commands/py.generate.cmd.ts
+13 -23
@@ -7,6 +7,7 @@ import { logMetrics } from 'src/effects/log-metrics';
7 import type { GetCmdParams } from 'src/type-utils';
8 import { NodeProcess } from 'src/services/node-process';
9 import { pyFindComposioCoreGenerated } from 'src/effects/find-composio-core-generated';
10 +import { generationOutcome, loadGenerationRuntime } from 'src/effects/generation-runtime';
11 import {
12 getToolkitVersionOverrides,
13 type ToolkitVersionOverrides,
@@ -58,23 +59,6 @@ const _pyCmd$Generate = Command.make('generate', { outputOpt, toolkitsOpt }).pip
59
60 export const pyCmd$Generate = _pyCmd$Generate.pipe(Command.withHandler(generatePythonTypeStubs));
61
61 -/**
62 - * Deferred for the same reason as the TypeScript pipeline: only
63 - * `composio generate` needs it, so the import stays inside the handler instead
64 - * of on every command's startup path.
65 - */
66 -const loadPythonGeneration = Effect.promise(() =>
67 - Promise.all([
68 - import('src/generation/create-toolkit-index'),
69 - import('src/generation/python/generate'),
70 - import('src/generation/constants'),
71 - ]).then(([toolkitIndex, generate, constants]) => ({
72 - createToolkitIndex: toolkitIndex.createToolkitIndex,
73 - generatePythonSources: generate.generatePythonSources,
74 - BANNER: constants.BANNER,
75 - }))
76 -);
77 -
62 export function generatePythonTypeStubs({
63 outputOpt,
64 toolkitsOpt,
@@ -185,14 +169,20 @@ export function generatePythonTypeStubs({
169 const typeableTools = { withTypes: false as const, tools };
170
171 yield* spinner.message('Generating Python type stubs...');
188 - const { createToolkitIndex, generatePythonSources, BANNER } = yield* loadPythonGeneration;
189 - const index = createToolkitIndex({ toolkits, typeableTools, triggerTypes, versionMap });
172 + // The generation pipeline lives in the `generation-runtime` companion
173 + // module, loaded from disk here so no other command pays for it at startup.
174 + const generation = yield* loadGenerationRuntime;
175 + const index = generation.createToolkitIndex({
176 + toolkits,
177 + typeableTools,
178 + triggerTypes,
179 + versionMap,
180 + });
181
182 // Generate Python sources
192 - const sources = yield* generatePythonSources({
193 - banner: BANNER,
194 - outputDir,
195 - })(index);
183 + const sources = yield* generationOutcome(() =>
184 + generation.generatePythonSourceFiles({ banner: generation.BANNER, outputDir }, index)
185 + );
186
187 yield* spinner.message('Writing files to disk...');
188
ts/packages/cli/src/commands/run.cmd.ts
+9 -8
@@ -5,6 +5,7 @@ import * as Path from 'effect/Path';
5 import { ChildProcess, ChildProcessSpawner } from 'effect/unstable/process';
6 import { Context, Data, Deferred, Duration, Effect, MutableRef, Option, Result } from 'effect';
7 import { APP_VERSION } from 'src/constants';
8 +import { loadGenerationRuntime } from 'src/effects/generation-runtime';
9 import { APP_CONFIG, UNPREFIXED_CONFIG } from 'src/effects/app-config';
10 import { resolveCommandProject } from 'src/services/command-project';
11 import { type RunHelperContext } from 'src/services/run-helpers-runtime';
@@ -105,15 +106,15 @@ export const RunPassthroughArgs = Context.Reference<ReadonlyArray<string> | unde
106 );
107
108 /**
108 - * `run-source-transforms` pulls in the TypeScript compiler, roughly 95ms of
109 - * module evaluation. Only `composio run` rewrites a script, so the import stays
110 - * inside the handler rather than on every command's startup path. A rejected
111 - * import of a module bundled into this binary is an impossible invariant, not a
112 - * recoverable failure, which is why this is `Effect.promise` and not
113 - * `Effect.tryPromise`. The module registry memoizes the import, so repeat calls
114 - * within one run cost nothing.
109 + * The source rewrites need the TypeScript compiler, which ships in the
110 + * `generation-runtime` companion module next to the executable rather than in
111 + * the executable itself. Loading it here keeps it off every other command's
112 + * startup path. A missing companion goes through the same self-repair as the
113 + * `run-*` modules, and its error surfaces exactly as theirs does below.
114 */
116 -const loadSourceTransforms = Effect.promise(() => import('./run-source-transforms'));
115 +const loadSourceTransforms = loadGenerationRuntime.pipe(
116 + Effect.mapError(error => new Error(error.message))
117 +);
118
119 export const inferCliInvocationPrefix = (
120 path: Path.Path,
ts/packages/cli/src/commands/tools/commands/tools.execute.cmd.ts
+12 -17
@@ -2,8 +2,6 @@ import { Argument, Command, Flag } from 'effect/unstable/cli';
2 import { isLocalToolSlug } from '@composio/cli-local-tools';
3 import util from 'node:util';
4 import { Cause, Data, Effect, Exit, Fiber, HashSet, Option, Result } from 'effect';
5 -import { Tiktoken } from 'js-tiktoken/lite';
6 -import o200kBase from 'js-tiktoken/ranks/o200k_base';
5 import { redact } from 'src/ui/redact';
6 import { parseJsonRecord, isPlainRecord } from 'src/utils/parse-json';
7 import { toolkitFromToolSlug } from 'src/effects/toolkit-from-tool-slug';
@@ -20,6 +18,7 @@ import {
18 } from 'src/services/tool-input-validation';
19 import { TerminalUI } from 'src/services/terminal-ui';
20 import { logToolDebug, makePerfDebugLogger } from 'src/services/runtime-debug-logger';
21 +import { loadInstalledCompanionModule } from 'src/services/run-companion-modules';
22 import {
23 LocalToolsDisabledError,
24 ToolsExecutor,
@@ -323,22 +322,18 @@ const redactRequestId = (value: object): object => {
322 };
323
324 const EXECUTE_INLINE_OUTPUT_TOKEN_THRESHOLD = 10_000;
326 -let executeOutputEncoder: Tiktoken | undefined;
325
328 -const getExecuteOutputEncoder = () => {
329 - if (!executeOutputEncoder) {
330 - executeOutputEncoder = new Tiktoken(o200kBase);
331 - }
332 - return executeOutputEncoder;
333 -};
326 +// The tokenizer lives in the `execute-output-encoder-runtime` companion module
327 +// next to the executable, loaded from disk on first use: its rank table is
328 +// 2.3MB that no other command, and no small response, has any use for. A
329 +// missing companion in a packaged install goes through the same self-repair as
330 +// `composio run`'s modules and fails the command the same way.
331 +const loadExecuteOutputEncoder = loadInstalledCompanionModule<
332 + typeof import('src/services/execute-output-encoder-runtime')
333 +>('execute-output-encoder-runtime');
334
335 -// `Tiktoken.encode` defaults `disallowedSpecial` to "all", which makes it throw
336 -// on any tool response that happens to contain the literal text `<|endoftext|>`
337 -// or `<|endofprompt|>` (a README about tokenizers is enough). Here the encoder
338 -// is only a length gauge, so passing `allowedSpecial: 'all'` counts each literal
339 -// as the single special token it encodes to instead of rejecting the payload.
340 -const countOutputTokens = (json: string): number =>
341 - getExecuteOutputEncoder().encode(json, 'all').length;
335 +const countOutputTokens = (json: string) =>
336 + Effect.map(loadExecuteOutputEncoder, encoder => encoder.countOutputTokens(json));
337
338 // A BPE token always covers at least one UTF-8 byte, so a payload of at most
339 // THRESHOLD bytes can never exceed THRESHOLD tokens. Checking the byte length
@@ -438,7 +433,7 @@ const prepareExecuteOutput = (
433 } satisfies PreparedExecuteOutput;
434 }
435
441 - const tokenCount = countOutputTokens(json);
436 + const tokenCount = yield* countOutputTokens(json);
437 if (tokenCount <= EXECUTE_INLINE_OUTPUT_TOKEN_THRESHOLD) {
438 return {
439 kind: 'inline',
ts/packages/cli/src/commands/ts/commands/ts.generate.cmd.ts
+25 -36
@@ -28,6 +28,7 @@ import { logMetrics } from 'src/effects/log-metrics';
28 import { NodeProcess } from 'src/services/node-process';
29 import type { GetCmdParams } from 'src/type-utils';
30 import { jsFindComposioCoreGenerated } from 'src/effects/find-composio-core-generated';
31 +import { generationOutcome, loadGenerationRuntime } from 'src/effects/generation-runtime';
32 import type { Toolkit } from 'src/models/toolkits';
33 import type { TriggerType } from 'src/models/trigger-types';
34 import type { Tool, ToolsAsEnums } from 'src/models/tools';
@@ -361,28 +362,6 @@ export const handleTsGenerate = (params: GetCmdParams<typeof _tsCmd$Generate>) =
362 });
363 };
364
364 -/**
365 - * The generation pipeline pulls in the TypeScript compiler, together roughly
366 - * 165ms of module evaluation, and only `composio generate` uses it. Importing it
367 - * from inside the handler keeps it off every other command's startup path. A
368 - * rejected import of a module bundled into this binary is an impossible
369 - * invariant rather than a recoverable failure, so this is `Effect.promise`. The
370 - * module registry memoizes the import.
371 - */
372 -const loadTypeScriptGeneration = Effect.promise(() =>
373 - Promise.all([
374 - import('src/generation/create-toolkit-index'),
375 - import('src/generation/typescript/generate'),
376 - import('src/generation/typescript/transpile'),
377 - import('src/generation/constants'),
378 - ]).then(([toolkitIndex, generate, transpile, constants]) => ({
379 - createToolkitIndex: toolkitIndex.createToolkitIndex,
380 - generateTypeScriptSources: generate.generateTypeScriptSources,
381 - transpileTypeScriptSources: transpile.transpileTypeScriptSources,
382 - BANNER: constants.BANNER,
383 - }))
384 -);
385 -
365 export function generateTypescriptTypeStubs({
366 outputOpt,
367 compact,
@@ -438,21 +417,29 @@ export function generateTypescriptTypeStubs({
417 : fetchAllData(client, typeTools, validatedOverrides, spinner);
418
419 yield* spinner.message('Generating TypeScript type stubs...');
441 - const {
442 - createToolkitIndex,
443 - generateTypeScriptSources,
444 - transpileTypeScriptSources,
445 - BANNER,
446 - } = yield* loadTypeScriptGeneration;
447 - const index = createToolkitIndex({ toolkits, typeableTools, triggerTypes, versionMap });
420 + // The generation pipeline and the TypeScript compiler live in the
421 + // `generation-runtime` companion module, loaded from disk here so no
422 + // other command pays for them at startup.
423 + const generation = yield* loadGenerationRuntime;
424 + const index = generation.createToolkitIndex({
425 + toolkits,
426 + typeableTools,
427 + triggerTypes,
428 + versionMap,
429 + });
430
431 // Generate TypeScript sources
450 - const sources = yield* generateTypeScriptSources({
451 - outputDir,
452 - emitSingleFile: Boolean(compact), // Ensure boolean type
453 - banner: BANNER,
454 - importExtension: 'js',
455 - })(index);
432 + const sources = yield* generationOutcome(() =>
433 + generation.generateTypeScriptSourceFiles(
434 + {
435 + outputDir,
436 + emitSingleFile: Boolean(compact), // Ensure boolean type
437 + banner: generation.BANNER,
438 + importExtension: 'js',
439 + },
440 + index
441 + )
442 + );
443
444 yield* spinner.message('Writing files to disk...');
445
@@ -479,7 +466,9 @@ export function generateTypescriptTypeStubs({
466 if (transpiled) {
467 yield* spinner.message('Transpiling to JavaScript...');
468 yield* pipe(
482 - transpileTypeScriptSources({ sources, outputDir }),
469 + generationOutcome(() =>
470 + generation.transpileTypeScriptSourceFiles({ sources, outputDir })
471 + ),
472 Effect.catch(error =>
473 Effect.logWarning(`Failed to compile TypeScript files: ${error.message}`)
474 )
ts/packages/cli/src/constants.ts
+12 -5
@@ -1,11 +1,18 @@
1 -import { constants } from '@composio/core';
1 import pkg from '../package.json' with { type: 'json' };
2
3 declare const __COMPOSIO_CLI_RELEASE_VERSION__: string | undefined;
4
6 -const { DEFAULT_BASE_URL, DEFAULT_WEB_URL } = constants;
5 +// These four values mirror `@composio/core`'s `constants` module. They are
6 +// spelled out here rather than imported because importing anything from the
7 +// core package's root entry evaluates the whole SDK at startup (its zod schemas
8 +// alone cost ~25ms) and pulls `openai` and `pusher-js` into the binary, for two
9 +// strings and two URLs. `test/src/constants.test.ts` pins them to core's values.
10
8 -export { DEFAULT_BASE_URL, DEFAULT_WEB_URL };
11 +/** Default base URL for the Composio API server (backend). */
12 +export const DEFAULT_BASE_URL = 'https://backend.composio.dev';
13 +
14 +/** Default base URL for the Composio web app (frontend). */
15 +export const DEFAULT_WEB_URL = 'https://dashboard.composio.dev/';
16
17 /**
18 * Staging base URL for the Composio API server (backend).
@@ -32,7 +39,7 @@ export const DEBUG_OVERRIDE_ENV_CONFIG_KEY_PREFIX = 'DEBUG_OVERRIDE_';
39 /**
40 * Name of the user config file used by the Composio CLI.
41 */
35 -export const USER_CONFIG_FILE_NAME = constants.USER_DATA_FILE_NAME;
42 +export const USER_CONFIG_FILE_NAME = 'user_data.json';
43
44 /**
45 * Name of the general CLI config file used by the Composio CLI.
@@ -42,7 +49,7 @@ export const CLI_CONFIG_FILE_NAME = 'config.json';
49 /**
50 * Name of the directory used to store the Composio CLI config.
51 */
45 -export const USER_COMPOSIO_DIR = constants.COMPOSIO_DIR;
52 +export const USER_COMPOSIO_DIR = '.composio';
53
54 /**
55 * A map of cache filenames used by the Composio CLI.
ts/packages/cli/src/effects/generation-runtime.ts new
+32
@@ -0,0 +1,32 @@
1 +import { Effect } from 'effect';
2 +import { rehydrateGenerationError, type GenerationError } from 'src/generation/errors';
3 +import { loadInstalledCompanionModule } from 'src/services/run-companion-modules';
4 +import type { GenerationOutcome } from 'src/services/generation-runtime';
5 +
6 +// Type-only reference: it leaves no import behind, so the compiler stays out of
7 +// the executable. The module itself is loaded from disk on first use.
8 +export type GenerationRuntime = typeof import('src/services/generation-runtime');
9 +
10 +/**
11 + * The `generation-runtime` companion module (TypeScript compiler, generation
12 + * pipeline, `composio run` source rewrites), loaded on demand. See
13 + * `src/services/generation-runtime.ts` for why its API is Effect-free.
14 + */
15 +export const loadGenerationRuntime =
16 + loadInstalledCompanionModule<GenerationRuntime>('generation-runtime');
17 +
18 +/**
19 + * Lifts a companion-module outcome into the CLI's runtime: successes succeed,
20 + * failures fail with the CLI's own instance of the error, and a rejected promise
21 + * (a defect inside the pipeline) stays a defect.
22 + */
23 +export const generationOutcome = <A>(
24 + run: () => Promise<GenerationOutcome<A>>
25 +): Effect.Effect<A, GenerationError> =>
26 + Effect.promise(run).pipe(
27 + Effect.flatMap(outcome =>
28 + outcome._tag === 'Success'
29 + ? Effect.succeed(outcome.value)
30 + : Effect.fail(rehydrateGenerationError(outcome.error))
31 + )
32 + );
ts/packages/cli/src/generation/errors.ts new
+63
@@ -0,0 +1,63 @@
1 +import { Data } from 'effect';
2 +
3 +// The generation pipeline runs inside the `generation-runtime` companion module,
4 +// a separately bundled file with its own copy of these classes. Errors cross
5 +// back into the CLI as plain values, so this module has no compiler import and
6 +// `rehydrateGenerationError` rebuilds them as instances of the CLI's classes,
7 +// which keeps `_tag` handling and error rendering exactly as before.
8 +
9 +export class SafeOutputPathError extends Data.TaggedError('generation/SafeOutputPathError')<{
10 + readonly filename: string;
11 + readonly outputDir: string;
12 + readonly resolvedPath: string;
13 + readonly message: string;
14 +}> {}
15 +
16 +export class TypeScriptTranspileError extends Data.TaggedError('error/TypeScriptTranspileError')<{
17 + readonly message: string;
18 + readonly cause: string;
19 +}> {}
20 +
21 +export class GenerateTypeFromJsonSchemaError extends Data.TaggedError('json-parsing-error')<{
22 + cause?: unknown;
23 + message?: string;
24 +}> {}
25 +
26 +export type GenerationError =
27 + SafeOutputPathError | TypeScriptTranspileError | GenerateTypeFromJsonSchemaError;
28 +
29 +const withOriginalStack = <E extends Error>(error: E, original: Error): E => {
30 + if (typeof original.stack === 'string') {
31 + Object.defineProperty(error, 'stack', { value: original.stack, configurable: true });
32 + }
33 + return error;
34 +};
35 +
36 +/**
37 + * Rebuilds a generation error produced by the companion module as an instance
38 + * of this module's class, carrying over its fields and stack.
39 + */
40 +export const rehydrateGenerationError = (error: GenerationError): GenerationError => {
41 + switch (error._tag) {
42 + case 'generation/SafeOutputPathError':
43 + return withOriginalStack(
44 + new SafeOutputPathError({
45 + filename: error.filename,
46 + outputDir: error.outputDir,
47 + resolvedPath: error.resolvedPath,
48 + message: error.message,
49 + }),
50 + error
51 + );
52 + case 'error/TypeScriptTranspileError':
53 + return withOriginalStack(
54 + new TypeScriptTranspileError({ message: error.message, cause: error.cause }),
55 + error
56 + );
57 + case 'json-parsing-error':
58 + return withOriginalStack(
59 + new GenerateTypeFromJsonSchemaError({ cause: error.cause, message: error.message }),
60 + error
61 + );
62 + }
63 +};
ts/packages/cli/src/generation/safe-output-path.ts
+3 -7
@@ -1,11 +1,7 @@
1 -import { Data, Effect, Path } from 'effect';
1 +import { Effect, Path } from 'effect';
2 +import { SafeOutputPathError } from 'src/generation/errors';
3
3 -export class SafeOutputPathError extends Data.TaggedError('generation/SafeOutputPathError')<{
4 - readonly filename: string;
5 - readonly outputDir: string;
6 - readonly resolvedPath: string;
7 - readonly message: string;
8 -}> {}
4 +export { SafeOutputPathError };
5
6 /**
7 * Joins a generated filename to the output directory, ensuring the result stays
ts/packages/cli/src/generation/typescript/generate-type-from-json-schema.ts
+2 -5
@@ -1,12 +1,9 @@
1 import ts from 'typescript';
2 import openapiTS, { type OpenAPI3, astToString } from 'openapi-typescript';
3 import { Effect } from 'effect';
4 -import { TaggedError } from 'effect/Data';
4 +import { GenerateTypeFromJsonSchemaError } from 'src/generation/errors';
5
6 -export class GenerateTypeFromJsonSchemaError extends TaggedError('json-parsing-error')<{
7 - cause?: unknown;
8 - message?: string;
9 -}> {}
6 +export { GenerateTypeFromJsonSchemaError };
7
8 export function generateTypeFromJsonSchema(
9 name: string,
ts/packages/cli/src/generation/typescript/transpile.ts
+3 -5
@@ -1,15 +1,13 @@
1 import ts from 'typescript';
2 -import { Effect, Data, Path } from 'effect';
2 +import { Effect, Path } from 'effect';
3 +import { TypeScriptTranspileError } from 'src/generation/errors';
4 import {
5 buildVirtualFileMap,
6 patchCompilerHostWithVirtualFiles,
7 formatDiagnostic,
8 } from './virtual-compiler-host';
9
9 -export class TypeScriptTranspileError extends Data.TaggedError('error/TypeScriptTranspileError')<{
10 - readonly message: string;
11 - readonly cause: string;
12 -}> {}
10 +export { TypeScriptTranspileError };
11
12 type TranspileTypeScriptFilesParams = {
13 sources: (readonly [filePath: string, content: string])[];
ts/packages/cli/src/services/execute-output-encoder-runtime.ts new
+31
@@ -0,0 +1,31 @@
1 +// Entry point of the `execute-output-encoder-runtime` companion module: the
2 +// tokenizer `composio execute` uses to decide whether a large response is
3 +// printed inline or stored to a file, bundled into
4 +// `dist/execute-output-encoder-runtime.mjs` next to the executable. The o200k
5 +// rank table is a 2.3MB string literal that would otherwise sit in the
6 +// executable for every command, while only responses past the inline threshold
7 +// ever need it. See `RUN_COMPANION_MODULE_BASENAMES`.
8 +//
9 +// The CLI loads this file with
10 +// `loadInstalledCompanionModule('execute-output-encoder-runtime')`. It exposes
11 +// plain functions on purpose; see `generation-runtime.ts` for why nothing
12 +// Effect-shaped crosses a companion boundary.
13 +import { Tiktoken } from 'js-tiktoken/lite';
14 +import o200kBase from 'js-tiktoken/ranks/o200k_base';
15 +
16 +let executeOutputEncoder: Tiktoken | undefined;
17 +
18 +const getExecuteOutputEncoder = () => {
19 + if (!executeOutputEncoder) {
20 + executeOutputEncoder = new Tiktoken(o200kBase);
21 + }
22 + return executeOutputEncoder;
23 +};
24 +
25 +// `Tiktoken.encode` defaults `disallowedSpecial` to "all", which makes it throw
26 +// on any tool response that happens to contain the literal text `<|endoftext|>`
27 +// or `<|endofprompt|>` (a README about tokenizers is enough). Here the encoder
28 +// is only a length gauge, so those literals are ordinary characters: passing
29 +// `allowedSpecial: 'all'` counts them instead of rejecting the payload.
30 +export const countOutputTokens = (json: string): number =>
31 + getExecuteOutputEncoder().encode(json, 'all').length;
ts/packages/cli/src/services/generation-runtime.ts new
+74
@@ -0,0 +1,74 @@
1 +// Entry point of the `generation-runtime` companion module: everything in the
2 +// CLI that needs the TypeScript compiler, bundled into `dist/generation-runtime.mjs`
3 +// next to the executable rather than into the executable itself. The compiler is
4 +// ~3.7MB of minified JavaScript that every invocation used to parse at startup
5 +// (about 74ms on the pinned toolchain) even though only `composio generate` and
6 +// `composio run` call it. See `RUN_COMPANION_MODULE_BASENAMES`.
7 +//
8 +// The CLI loads this file with `loadInstalledCompanionModule('generation-runtime')`.
9 +// Because the bundle carries its own copy of `effect`, nothing Effect-shaped may
10 +// cross the boundary: a fiber cannot run primitives built by another copy of the
11 +// runtime. So this module exposes plain functions and promises, runs the Effect
12 +// pipelines on its own runtime, and reports failures as values that
13 +// `rehydrateGenerationError` turns back into the CLI's error classes. Results
14 +// stay object references (same process), no serialization happens.
15 +import * as BunPath from '@effect/platform-bun/BunPath';
16 +import type * as Path from 'effect/Path';
17 +import { Effect, Result } from 'effect';
18 +import type { ToolkitIndex } from 'src/generation/create-toolkit-index';
19 +import type { GenerationError } from 'src/generation/errors';
20 +import { generatePythonSources } from 'src/generation/python/generate';
21 +import type { SourceFile } from 'src/generation/types';
22 +import { generateTypeScriptSources } from 'src/generation/typescript/generate';
23 +import { transpileTypeScriptSources } from 'src/generation/typescript/transpile';
24 +
25 +export { BANNER } from 'src/generation/constants';
26 +export { createToolkitIndex } from 'src/generation/create-toolkit-index';
27 +export type { ToolkitIndex } from 'src/generation/create-toolkit-index';
28 +export {
29 + extractInlineExecuteToolSlugs,
30 + wrapFileSourceForRun,
31 + wrapInlineCodeForRun,
32 +} from 'src/commands/run-source-transforms';
33 +
34 +/** Outcome of a generation step, as a value the CLI's own runtime can consume. */
35 +export type GenerationOutcome<A> =
36 + | { readonly _tag: 'Success'; readonly value: A }
37 + | { readonly _tag: 'Failure'; readonly error: GenerationError };
38 +
39 +// Failures become `GenerationOutcome`; a defect (an unexpected throw inside the
40 +// pipeline) still rejects the promise, so the CLI treats it as a defect too.
41 +const runOutcome = <A, E extends GenerationError>(
42 + effect: Effect.Effect<A, E, Path.Path>
43 +): Promise<GenerationOutcome<A>> =>
44 + Effect.runPromise(
45 + effect.pipe(
46 + Effect.provide(BunPath.layer),
47 + Effect.result,
48 + Effect.map((result): GenerationOutcome<A> =>
49 + Result.isSuccess(result)
50 + ? { _tag: 'Success', value: result.success }
51 + : { _tag: 'Failure', error: result.failure }
52 + )
53 + )
54 + );
55 +
56 +export type GenerateTypeScriptSourcesParams = Parameters<typeof generateTypeScriptSources>[0];
57 +export type TranspileTypeScriptSourcesParams = Parameters<typeof transpileTypeScriptSources>[0];
58 +export type GeneratePythonSourcesParams = Parameters<typeof generatePythonSources>[0];
59 +
60 +export const generateTypeScriptSourceFiles = (
61 + params: GenerateTypeScriptSourcesParams,
62 + index: ToolkitIndex
63 +): Promise<GenerationOutcome<Array<SourceFile>>> =>
64 + runOutcome(generateTypeScriptSources(params)(index));
65 +
66 +export const transpileTypeScriptSourceFiles = (
67 + params: TranspileTypeScriptSourcesParams
68 +): Promise<GenerationOutcome<void>> => runOutcome(transpileTypeScriptSources(params));
69 +
70 +export const generatePythonSourceFiles = (
71 + params: GeneratePythonSourcesParams,
72 + index: ToolkitIndex
73 +): Promise<GenerationOutcome<Array<SourceFile>>> =>
74 + runOutcome(generatePythonSources(params)(index));
ts/packages/cli/src/services/run-companion-modules.ts
+65 -7
@@ -6,7 +6,7 @@ import * as FileSystem from 'effect/FileSystem';
6 import * as Path from 'effect/Path';
7 import { Config, ConfigProvider, Data, Effect, Option, PlatformError, Schema } from 'effect';
8 import { extractZipSafely } from 'src/utils/extract-zip-safely';
9 -import { IS_RELEASE_BUILD } from 'src/constants';
9 +import { APP_VERSION, IS_RELEASE_BUILD } from 'src/constants';
10 import { GitHubRelease } from 'src/effects/resolve-cli-release';
11 import { getBaseConfigProvider, extendConfigProvider } from 'src/services/config';
12 import { NodeOs } from 'src/services/node-os';
@@ -14,12 +14,22 @@ import { atomicReplaceFile } from 'src/utils/atomic-replace';
14 import { parseChecksumsText, sha256Hex } from 'src/utils/checksums';
15 import { CLI_RELEASE_TAG_PREFIX } from 'src/utils/cli-release-version';
16
17 +// Modules the binary build bundles separately (`dist/<name>.mjs` next to the
18 +// executable) instead of into the executable itself. The first five are what
19 +// `composio run` preloads into the script it spawns. The last two are loaded
20 +// into the CLI's own process, on demand, through `loadInstalledCompanionModule`:
21 +// they carry the TypeScript compiler and the tokenizer rank table, which
22 +// together were ~70% of the executable's JavaScript and cost every command
23 +// parse time even though only `generate`, `run`, and large `execute` responses
24 +// ever reach them.
25 export const RUN_COMPANION_MODULE_BASENAMES: ReadonlyArray<string> = [
26 'run-helpers-runtime',
27 'run-subagent-shared',
28 'run-subagent-acp',
29 'run-subagent-legacy',
30 'run-subagent-output-mcp',
31 + 'generation-runtime',
32 + 'execute-output-encoder-runtime',
33 ];
34
35 export const RUN_COMPANION_MODULE_FILENAMES = RUN_COMPANION_MODULE_BASENAMES.map(
@@ -439,10 +449,11 @@ export const writeInstalledReleaseTag = (
449 });
450
451 /**
442 - * Startup tier: the `run-*.mjs` companion wrappers and their import graph.
452 + * Startup tier: the companion wrappers and their import graph.
453 *
444 - * Every `composio run` preloads these into the spawned child, so a missing one
445 - * really is a broken install and justifies the self-repair download. The ACP
454 + * Every `composio run` preloads the `run-*` modules into the spawned child, and
455 + * the CLI loads the in-process ones on demand, so a missing one really is a
456 + * broken install and justifies the self-repair download. The ACP
457 * adapter assets are deliberately excluded — a script like
458 * `composio run 'console.log(1)'` never invokes a sub-agent, and requiring
459 * ~224MB of adapters for it turned a working install into a hard failure.
@@ -672,7 +683,7 @@ export const repairMissingInstalledRunCompanionModules = ({
683 catch: error =>
684 new RunCompanionRepairError({
685 message: [
675 - `Unable to restore the files required by 'composio run' for ${releaseTag}.`,
686 + `Unable to restore the CLI's bundled support files for ${releaseTag}.`,
687 error instanceof Error ? error.message : String(error),
688 `Reinstall the CLI, or set GITHUB_TAG to the exact release tag for this build and try again.`,
689 ].join('\n'),
@@ -761,7 +772,7 @@ export const repairMissingInstalledRunCompanionModules = ({
772 if (!sourceExists) {
773 return yield* Effect.fail(
774 new RunCompanionRepairError({
764 - message: `Release ${release.tag_name} is missing ${relativePath}; cannot restore the files required by 'composio run'.`,
775 + message: `Release ${release.tag_name} is missing ${relativePath}; cannot restore the CLI's bundled support files.`,
776 })
777 );
778 }
@@ -773,7 +784,7 @@ export const repairMissingInstalledRunCompanionModules = ({
784 error =>
785 new RunCompanionRepairError({
786 message: [
776 - `Unable to restore the files required by 'composio run' for ${releaseTag}.`,
787 + `Unable to restore the CLI's bundled support files for ${releaseTag}.`,
788 error.message,
789 `Reinstall the CLI, or set GITHUB_TAG to the exact release tag for this build and try again.`,
790 ].join('\n'),
@@ -827,3 +838,50 @@ export const resolveRunCompanionModulePath = ({
838 : path.resolve(currentDirectory, `${baseName}.mjs`)
839 );
840 });
841 +
842 +/**
843 + * Loads one of the in-process companion modules (see the note on
844 + * `RUN_COMPANION_MODULE_BASENAMES`) and returns its exports.
845 + *
846 + * Resolution is the same as for the `run` companions: the `.ts` source next to
847 + * this file when running from a checkout (tests, `bun run src/bin.ts`), the
848 + * `.mjs` bundle next to the executable in a packaged install. A packaged install
849 + * that lost the file goes through the same release-archive repair `composio run`
850 + * uses before failing.
851 + *
852 + * The specifier is computed at runtime on purpose: a literal `import('./x')`
853 + * would make the bundler fold the module back into the executable, which is the
854 + * exact thing these modules exist to avoid.
855 + *
856 + * `M` is the module's type; pass `typeof import('src/services/<name>')`, which
857 + * is type-only and leaves no import behind.
858 + */
859 +export const loadInstalledCompanionModule = <M>(
860 + baseName: string
861 +): Effect.Effect<
862 + M,
863 + RunCompanionRepairError | PlatformError.PlatformError,
864 + FileSystem.FileSystem | Path.Path
865 +> =>
866 + Effect.gen(function* () {
867 + const path = yield* Path.Path;
868 + const callerImportMetaUrl = import.meta.url;
869 + const execPath = process.execPath;
870 +
871 + yield* repairMissingInstalledRunCompanionModules({
872 + callerImportMetaUrl,
873 + execPath,
874 + appVersion: APP_VERSION,
875 + });
876 +
877 + const modulePath = yield* resolveRunCompanionModulePath({
878 + callerImportMetaUrl,
879 + execPath,
880 + relativeNoExtensionFromCaller: `./${baseName}`,
881 + });
882 + const moduleUrl = yield* Effect.orDie(path.toFileUrl(modulePath));
883 +
884 + // A rejected import here means the file resolved above is unloadable, which
885 + // is a broken install rather than a recoverable failure.
886 + return yield* Effect.promise(() => import(moduleUrl.href) as Promise<M>);
887 + });
ts/packages/cli/src/services/tool-file-uploads.ts
+11 -2
@@ -6,13 +6,18 @@
6 import type * as FileSystem from 'effect/FileSystem';
7 import type * as Path from 'effect/Path';
8 import type { Composio as RawComposioClient } from '@composio/client';
9 -import { assertSafeFileUploadPath, readResponseBodyWithLimit } from '@composio/core';
10 -import { ssrfSafeFetch } from '@composio/core/utils/ssrf-guard';
9 import { Cause, Data, Effect, Exit, Predicate } from 'effect';
10 import { guessToolkitFromToolSlug } from 'src/utils/toolkit-from-tool-slug';
11
12 type JsonSchema = Record<string, unknown>;
13
14 +// The three core helpers below are only reached once an execute actually carries
15 +// a file input, so they are imported on that path rather than at module scope:
16 +// `@composio/core`'s root entry evaluates the whole SDK, which every command
17 +// would otherwise pay for at startup. The module registry memoizes the import.
18 +const loadCoreFileUploadHelpers = () => import('@composio/core');
19 +const loadSsrfSafeFetch = () => import('@composio/core/utils/ssrf-guard');
20 +
21 export class ToolFileUploadError extends Data.TaggedError('services/ToolFileUploadError')<{
22 readonly cause?: unknown;
23 readonly message: string;
@@ -145,6 +150,7 @@ export const findFileUploadablePaths = (
150 };
151
152 const readFileFromUrl = async (path: Path.Path, url: string) => {
153 + const { ssrfSafeFetch } = await loadSsrfSafeFetch();
154 const response = await ssrfSafeFetch(url);
155 if (!response.ok) {
156 await response.body?.cancel().catch(() => undefined);
@@ -158,6 +164,7 @@ const readFileFromUrl = async (path: Path.Path, url: string) => {
164 // Same cap as the core SDK's URL uploads: a remote server must not be able
165 // to exhaust memory with an oversized or never-ending body. Copied into a
166 // fresh ArrayBuffer-backed view for the same reason as readFileFromDisk.
167 + const { readResponseBodyWithLimit } = await loadCoreFileUploadHelpers();
168 const bytes = new Uint8Array(await readResponseBodyWithLimit(response));
169 const parsedUrl = new URL(url);
170 const fileName = path.basename(parsedUrl.pathname) || `file-${Date.now()}`;
@@ -193,6 +200,7 @@ const readFileFromDisk = async (fs: FileSystem.FileSystem, path: Path.Path, file
200 // agent that has been prompt-injected into supplying its own tool arguments, so a
201 // `--force`/env override would hand that attacker a trivial bypass. Pass a
202 // CLI-appropriate remediation so the error does not advertise an SDK-only opt-out.
203 + const { assertSafeFileUploadPath } = await loadCoreFileUploadHelpers();
204 assertSafeFileUploadPath(filePath, {
205 remediation:
206 'The Composio CLI always enforces this denylist and has no opt-out. To upload this ' +
@@ -255,6 +263,7 @@ const uploadFile = async (params: {
263 toolkit_slug: params.toolkitSlug,
264 });
265
266 + const { ssrfSafeFetch } = await loadSsrfSafeFetch();
267 const uploadResponse = await ssrfSafeFetch(presigned.new_presigned_url, {
268 method: 'PUT',
269 body: fileData.bytes,
ts/packages/cli/test/src/constants.test.ts new
+20
@@ -0,0 +1,20 @@
1 +import { constants } from '@composio/core';
2 +import { describe, expect, it } from 'vitest';
3 +import {
4 + DEFAULT_BASE_URL,
5 + DEFAULT_WEB_URL,
6 + USER_COMPOSIO_DIR,
7 + USER_CONFIG_FILE_NAME,
8 +} from 'src/constants';
9 +
10 +// `src/constants.ts` spells these out instead of importing them so the CLI never
11 +// evaluates the core SDK's root entry at startup. This is the only place the two
12 +// copies meet: if core changes a value, this fails instead of the CLI drifting.
13 +describe('constants mirrored from @composio/core', () => {
14 + it('match the core SDK values', () => {
15 + expect(DEFAULT_BASE_URL).toBe(constants.DEFAULT_BASE_URL);
16 + expect(DEFAULT_WEB_URL).toBe(constants.DEFAULT_WEB_URL);
17 + expect(USER_CONFIG_FILE_NAME).toBe(constants.USER_DATA_FILE_NAME);
18 + expect(USER_COMPOSIO_DIR).toBe(constants.COMPOSIO_DIR);
19 + });
20 +});
ts/packages/cli/test/src/effects/generation-runtime.test.ts new
+93
@@ -0,0 +1,93 @@
1 +import { describe, expect, it } from '@effect/vitest';
2 +import { Cause, Effect, Exit } from 'effect';
3 +import { generationOutcome } from 'src/effects/generation-runtime';
4 +import {
5 + GenerateTypeFromJsonSchemaError,
6 + rehydrateGenerationError,
7 + SafeOutputPathError,
8 + TypeScriptTranspileError,
9 +} from 'src/generation/errors';
10 +
11 +// The companion module ships its own copy of the error classes, so what comes
12 +// back is an instance of a class the CLI has never seen: same tag, same fields,
13 +// same stack, but `instanceof` the CLI's class is false. This clone reproduces
14 +// exactly that.
15 +const foreign = <E extends Error & { readonly _tag: string }>(error: E): E => {
16 + const clone = Object.create(Error.prototype) as E;
17 + for (const key of Object.getOwnPropertyNames(error)) {
18 + Object.defineProperty(clone, key, Object.getOwnPropertyDescriptor(error, key)!);
19 + }
20 + // `stack` is a lazy accessor bound to the original instance; a real foreign
21 + // instance owns its own, so give the clone a plain copy.
22 + Object.defineProperty(clone, 'stack', { value: error.stack, configurable: true });
23 + Object.defineProperty(clone, '_tag', { value: error._tag, enumerable: true });
24 + return clone;
25 +};
26 +
27 +describe('generation-runtime', () => {
28 + describe('rehydrateGenerationError', () => {
29 + it('rebuilds each error as an instance of the CLI class, keeping fields and stack', () => {
30 + const cases = [
31 + new SafeOutputPathError({
32 + filename: '../x.ts',
33 + outputDir: '/out',
34 + resolvedPath: '/x.ts',
35 + message: 'escapes',
36 + }),
37 + new TypeScriptTranspileError({ message: 'compile failed', cause: 'TS1005' }),
38 + new GenerateTypeFromJsonSchemaError({ message: 'bad schema', cause: new Error('inner') }),
39 + ] as const;
40 +
41 + for (const original of cases) {
42 + const crossed = foreign(original);
43 + expect(crossed).not.toBeInstanceOf(original.constructor);
44 +
45 + const rehydrated = rehydrateGenerationError(crossed);
46 + expect(rehydrated).toBeInstanceOf(original.constructor);
47 + expect(rehydrated._tag).toBe(original._tag);
48 + expect(rehydrated.message).toBe(original.message);
49 + expect(rehydrated.stack).toBe(original.stack);
50 + }
51 + });
52 + });
53 +
54 + describe('generationOutcome', () => {
55 + it.effect('lifts a success outcome to the value', () =>
56 + Effect.gen(function* () {
57 + const value = yield* generationOutcome(() =>
58 + Promise.resolve({ _tag: 'Success' as const, value: 42 })
59 + );
60 + expect(value).toBe(42);
61 + })
62 + );
63 +
64 + it.effect('lifts a failure outcome to a typed failure of the CLI error class', () =>
65 + Effect.gen(function* () {
66 + const error = new TypeScriptTranspileError({ message: 'compile failed', cause: 'TS1005' });
67 + const exit = yield* Effect.exit(
68 + generationOutcome(() => Promise.resolve({ _tag: 'Failure' as const, error }))
69 + );
70 +
71 + expect(Exit.isFailure(exit)).toBe(true);
72 + if (Exit.isFailure(exit)) {
73 + const failure = Cause.squash(exit.cause);
74 + expect(failure).toBeInstanceOf(TypeScriptTranspileError);
75 + expect((failure as TypeScriptTranspileError).cause).toBe('TS1005');
76 + }
77 + })
78 + );
79 +
80 + it.effect('keeps a rejected promise (a defect in the pipeline) as a defect', () =>
81 + Effect.gen(function* () {
82 + const boom = new Error('boom');
83 + const exit = yield* Effect.exit(generationOutcome<number>(() => Promise.reject(boom)));
84 +
85 + expect(Exit.isFailure(exit)).toBe(true);
86 + if (Exit.isFailure(exit)) {
87 + expect(Cause.hasDies(exit.cause)).toBe(true);
88 + expect(Cause.squash(exit.cause)).toBe(boom);
89 + }
90 + })
91 + );
92 + });
93 +});
ts/packages/cli/test/src/services/run-companion-modules.test.ts
+45 -2
@@ -8,10 +8,12 @@ import { vi } from 'vitest';
8 import {
9 hostRunCompanionStaticAssetRelativePaths,
10 listMissingInstalledRunCompanionModules,
11 + loadInstalledCompanionModule,
12 repairMissingInstalledRunCompanionModules,
13 resolveRunCompanionAssetPath,
14 RUN_CODEX_ACP_BINARY_TARGETS,
15 RUN_COMPANION_ALL_STATIC_ASSET_RELATIVE_PATHS,
16 + RUN_COMPANION_MODULE_BASENAMES,
17 RUN_COMPANION_MODULE_FILENAMES,
18 RUN_COMPANION_RELEASE_TAG_FILENAME,
19 RUN_COMPANION_SHARED_STATIC_ASSET_RELATIVE_PATHS,
@@ -250,7 +252,7 @@ describe('run-companion-modules', () => {
252 }).pipe(Effect.flip);
253
254 expect(error.message).toContain(
253 - `missing ${missingRelativePath}; cannot restore the files required by 'composio run'`
255 + `missing ${missingRelativePath}; cannot restore the CLI's bundled support files`
256 );
257 expect(fs.readFileSync(releaseTagPath, 'utf8')).toBe(previousReleaseTag);
258 }).pipe(
@@ -281,7 +283,7 @@ describe('run-companion-modules', () => {
283 appVersion: '0.0.0-test',
284 }).pipe(Effect.flip);
285
284 - expect(error.message).toContain("Unable to restore the files required by 'composio run'");
286 + expect(error.message).toContain("Unable to restore the CLI's bundled support files");
287 expect(fetchMock).toHaveBeenCalledOnce();
288 }).pipe(
289 Effect.ensuring(
@@ -453,3 +455,44 @@ describe('resolveRunCompanionAssetPath', () => {
455 );
456 });
457 });
458 +
459 +describe('loadInstalledCompanionModule', () => {
460 + layer(BunServices.layer)(it => {
461 + it('registers the in-process companions alongside the run preload set', () => {
462 + expect(RUN_COMPANION_MODULE_BASENAMES).toEqual(
463 + expect.arrayContaining(['generation-runtime', 'execute-output-encoder-runtime'])
464 + );
465 + });
466 +
467 + // From a checkout there is no `.mjs` next to an executable, so the loader
468 + // has to fall through to the `.ts` source next to `run-companion-modules.ts`.
469 + // The compiled-binary path (`dist/<name>.mjs` next to `process.execPath`)
470 + // is covered by the Docker E2E suite, which runs the real binary.
471 + it.effect('[Given] a source checkout [Then] it loads the module from its .ts source', () =>
472 + Effect.gen(function* () {
473 + const encoder = yield* loadInstalledCompanionModule<
474 + typeof import('src/services/execute-output-encoder-runtime')
475 + >('execute-output-encoder-runtime');
476 +
477 + expect(encoder.countOutputTokens('hello world')).toBe(2);
478 + // Special-token literals are counted as text rather than rejected.
479 + expect(encoder.countOutputTokens('<|endoftext|>')).toBeGreaterThan(0);
480 + })
481 + );
482 +
483 + it.effect('[Given] the generation companion [Then] its API is promise-shaped', () =>
484 + Effect.gen(function* () {
485 + const generation =
486 + yield* loadInstalledCompanionModule<typeof import('src/services/generation-runtime')>(
487 + 'generation-runtime'
488 + );
489 +
490 + expect(generation.wrapInlineCodeForRun('1 + 1')).toBe('return (1 + 1);');
491 + const outcome = yield* Effect.promise(() =>
492 + generation.generatePythonSourceFiles({ banner: 'b', outputDir: '/tmp/out' }, {})
493 + );
494 + expect(outcome).toEqual({ _tag: 'Success', value: [] });
495 + })
496 + );
497 + });
498 +});