@devries-cameron20 / composio Mirror · read-only / commits / ebe8bb780
Fork 0

Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

Mirrored from https://github.com/Camo67/composio.git

Synced Nov 21, 12:15 UTC

perf(cli): cut 221ms and 44MB RSS off every CLI invocation

A compiled Bun binary parses its whole embedded module graph before the first line of JS runs, so bundled-but-unused code is paid for on every invocation. Verified: a binary that bundles everything but evaluates only console.log still costs ~235ms, against 15ms for a hello-world build. Most of this bundle was code `composio execute` never reaches. Measured on the pinned toolchain (Bun 1.4.1+4661e494f, linux-x64, best of 7, analytics disabled): composio --version 970ms -> 749ms (-221ms) peak RSS 175.8MB -> 132.2MB (-43.6MB) compiled binary 96MB -> 86MB bundle 31.24MB -> 15.84MB Four changes. run.cmd.ts imported `ts` from ts-morph, which vendors its own copy of the TypeScript compiler, so the binary carried two of them. The file uses only createSourceFile, forEachChild, ScriptTarget, ScriptKind and five isX guards, all available in the typescript copy that src/generation/typescript/* already pulls in. Sharing one compiler also means commands/generate and commands/run.cmd no longer evaluate a compiler each. js-tiktoken's main entry statically inlines all six BPE rank tables (gpt2, r50k, p50k, p50k_edit, cl100k, o200k) as string literals; the CLI only ever uses o200k, via encodingForModel('gpt-4o'). The lite build with that single table produces identical token-id streams. prepareExecuteOutput built the o200k rank table on every successful execute, purely to compare the response against a 10k-token threshold. Constructing it measured ~390ms in a compiled binary on the pinned toolchain (390.1, 367.1, 418.6ms across three runs), against ~4ms to encode a 7.5KB payload once the table exists. A BPE token always covers at least one UTF-8 byte, so a payload of at most THRESHOLD bytes can never exceed THRESHOLD tokens; checking byte length first reaches the same decision without the tokenizer. That ~390ms is construction cost measured in isolation, not an end-to-end delta on a real `composio execute`. No credentialed run was available to measure the whole command before and after, so treat it as the size of the work removed from the success path rather than a verified wall-clock saving. `COMPOSIO_PERF_DEBUG=1` reports the gap between `execute.tool_call.end` and exit for anyone able to run it for real. ToolsExecutorLive resolved a client through clientSingleton.get() unconditionally, walking the project context off disk, then discarded it because every caller on the remote-execute path passes one in. Three behavioral deltas, none of them the tokenization result or the inline/file decision: 1. Tiktoken.encode() throws on the literals <|endoftext|> and <|endofprompt|> appearing anywhere in the response, at any size (a 66-byte payload reproduces it). That throw landed after "Execution successful" had printed, so the tool ran and the CLI still exited 1 with nothing on stdout. Responses at or under 10KB no longer reach encode(), so they now succeed. Larger responses still hit it; the real fix is passing allowedSpecial 'all' and is not in this commit. 2. TypeScript 6.0.2 (ts-morph's vendored copy) to 6.0.3. Differential tested: the three real parse helpers over 20 sources covering TSX, decorators, `using`, `satisfies`, import attributes, optional-chained calls and unicode gave identical output on all 60 comparisons. 3. Under COMPOSIO_LOG_LEVEL=Debug, ProjectContext's "resolved from ..." debug lines no longer appear on the remote-execute path. The local-tool path still calls get() and is unchanged. Verified with typecheck:src, oxlint, validate:boundaries, the tools.execute and run command suites (89 tests), and differential tests of both tokenizers and both TypeScript versions. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EzaE7oGVgziJ5nRvBhcci2

Claude committed Sep 12, 2026 at 18:46 UTC ebe8bb780f0d1c6fae6eca47060035827f2ba221
3 files changed +19 -11
ts/packages/cli/src/commands/run.cmd.ts
+1 -1
@@ -4,7 +4,7 @@ import * as FileSystem from 'effect/FileSystem';
4 import * as Path from 'effect/Path';
5 import { ChildProcess, ChildProcessSpawner } from 'effect/unstable/process';
6 import { Context, Data, Deferred, Duration, Effect, MutableRef, Option, Result } from 'effect';
7 -import { ts } from 'ts-morph';
7 +import ts from 'typescript';
8 import { APP_VERSION } from 'src/constants';
9 import { APP_CONFIG, UNPREFIXED_CONFIG } from 'src/effects/app-config';
10 import { resolveCommandProject } from 'src/services/command-project';
ts/packages/cli/src/commands/tools/commands/tools.execute.cmd.ts
+14 -8
@@ -2,7 +2,8 @@ import { Argument, Command, Flag } from 'effect/unstable/cli';
2 import { isLocalToolSlug } from '@composio/cli-local-tools';
3 import util from 'node:util';
4 import { Cause, Data, Effect, Exit, Fiber, HashSet, Option, Result } from 'effect';
5 -import { encodingForModel } from 'js-tiktoken';
5 +import { Tiktoken } from 'js-tiktoken/lite';
6 +import o200kBase from 'js-tiktoken/ranks/o200k_base';
7 import { redact } from 'src/ui/redact';
8 import { parseJsonRecord, isPlainRecord } from 'src/utils/parse-json';
9 import { toolkitFromToolSlug } from 'src/effects/toolkit-from-tool-slug';
@@ -322,15 +323,24 @@ const redactRequestId = (value: object): object => {
323 };
324
325 const EXECUTE_INLINE_OUTPUT_TOKEN_THRESHOLD = 10_000;
325 -let executeOutputEncoder: ReturnType<typeof encodingForModel> | undefined;
326 +let executeOutputEncoder: Tiktoken | undefined;
327
328 const getExecuteOutputEncoder = () => {
329 if (!executeOutputEncoder) {
329 - executeOutputEncoder = encodingForModel('gpt-4o');
330 + executeOutputEncoder = new Tiktoken(o200kBase);
331 }
332 return executeOutputEncoder;
333 };
334
335 +// A BPE token always covers at least one UTF-8 byte, so a payload of at most
336 +// THRESHOLD bytes can never exceed THRESHOLD tokens. Checking the byte length
337 +// first keeps the common (small) response off the tokenizer entirely: building
338 +// the o200k rank table measured ~390ms in a compiled binary, against ~4ms to
339 +// encode a 7.5KB payload once it exists, and microseconds to measure the bytes.
340 +const exceedsInlineOutputThreshold = (json: string): boolean =>
341 + new TextEncoder().encode(json).length > EXECUTE_INLINE_OUTPUT_TOKEN_THRESHOLD &&
342 + getExecuteOutputEncoder().encode(json).length > EXECUTE_INLINE_OUTPUT_TOKEN_THRESHOLD;
343 +
344 const shouldStoreLargeExecuteOutput = APP_CONFIG.CLI_INVOCATION_ORIGIN.pipe(
345 Effect.orDie,
346 Effect.map(origin => origin !== 'run')
@@ -407,11 +417,7 @@ const prepareExecuteOutput = (
417 ) =>
418 Effect.gen(function* () {
419 const json = serializeExecuteOutput(result);
410 - const tokenCount = getExecuteOutputEncoder().encode(json).length;
411 - if (
412 - tokenCount <= EXECUTE_INLINE_OUTPUT_TOKEN_THRESHOLD ||
413 - !(yield* shouldStoreLargeExecuteOutput)
414 - ) {
420 + if (!exceedsInlineOutputThreshold(json) || !(yield* shouldStoreLargeExecuteOutput)) {
421 return {
422 kind: 'inline',
423 json,
ts/packages/cli/src/services/tools-executor.ts
+4 -2
@@ -172,8 +172,10 @@ export const ToolsExecutorLive = Layer.effect(
172 }
173 }
174
175 - const client = yield* clientSingleton.get();
176 - const resolvedClient = params.client ?? client;
175 + // Resolved lazily: `get()` walks the project context off disk, and every
176 + // caller on the execute path already hands in a client built for the
177 + // resolved org/project.
178 + const resolvedClient = params.client ?? (yield* clientSingleton.get());
179 // One session per invocation — CLI runs one tool per process.
180 const {
181 sessionId,