@devries-cameron20 / composio-1 / commits / 9b1cbb180
Fork 0

Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

perf(cli): move the compiler and tokenizer out of the executable (#4469)

## Summary `composio --version`: 288ms to 199ms. Peak RSS: 97.8MB to 77.3MB. Executable: 85.9MB to 79.7MB. Every command benefits. A compiled Bun binary parses its whole embedded bundle before running any JS. #4468 stopped the TypeScript compiler and the tokenizer rank table from being evaluated at startup, but they were still parsed every time. The compiler was 44% of the executable's JavaScript, the o200k table another 28%. Both now ship as files next to the executable and load on demand. Fourth PR in the stack. Stacked on #4468; review #4463, #4464 and #4468 first. #4475 builds on this one. Bun 1.4.1+4661e494f, linux-x64, best of 15, telemetry disabled, both binaries built in the same session: | | before (#4468) | after | |---|---|---| | `composio --version` | 288ms | 199ms | | `composio tools execute --help` | 287ms | 202ms | | peak RSS | 97.8MB | 77.3MB | | executable | 85.9MB | 79.7MB | | executable JS, minified | 8.3MB | 2.1MB | Across the whole stack, from `next`: `--version` 612ms to 184ms, peak RSS 167MB to 78MB, executable 95.8MB to 79.7MB. `composio execute` end to end, against the live backend with a logged-in CLI, best of 7 for the small response and best of 5 for the large one. Tool: `HACKERNEWS_GET_ITEM_WITH_ID` (no connected account needed) and `HACKERNEWS_GET_LATEST_POSTS`. "Tail" is the time from the `execute.tool_call.end` perf event to process exit. | | `next` | #4468 | this PR | |---|---|---|---| | 1.6KB response, wall | 2431ms | 1857ms | 1761ms | | 1.6KB response, tail | 294ms | 12ms | 11ms | | 35KB response, wall | 2665ms | 2210ms | 2165ms | | 35KB response, tail | 322ms | 329ms | 353ms | The stack removes ~670ms from a small execute: ~430ms of startup and ~280ms of tokenizer construction that no longer happens. The large response keeps its ~330ms tail because past 10KB the tokenizer is still built; this PR adds ~20ms there for the on-demand parse of the encoder file. The remaining ~1.7s is network the stack does not touch: DNS and TLS to the backend, the preflight round trips before `tool_call.start`, and the session create plus execute pair. Wall times move by ±150ms between runs because of that; the tail column is the stable one. ## Changes 1. `generation-runtime.mjs` carries `src/generation/*`, the `composio run` source rewrites, `typescript`, `@composio/ts-builders` and `openapi-typescript`. `generate ts`, `generate py` and `run` load it with the new `loadInstalledCompanionModule`. From a source checkout the loader resolves the `.ts` file next to `run-companion-modules.ts`, so tests and `bun run src/bin.ts` need no build step. The specifier is computed at runtime on purpose; a literal `import('./x')` gets folded back into the executable. Before importing, a packaged install runs the self-repair download only if that companion's own files (its wrapper and what the wrapper imports) are missing, so a different missing file cannot block it. The repair has to come first, because Bun keeps a failed or already-loaded import in its module registry. A file that fails to import, or lacks one of the exports its caller names, is a typed `RunCompanionRepairError` asking to reinstall, not a crash. Both companions are also tsdown entries, so the `dist/` build resolves them. 2. `execute-output-encoder-runtime.mjs` carries `js-tiktoken/lite` plus the rank table. `execute` loads it only past the 10KB byte gate from #4463, and never for executes started by `composio run`. If it cannot be loaded, even after the self-repair download, `execute` estimates the token count from the byte length (about four bytes per token) instead of failing a tool call that already succeeded. The estimate can undercount, so such a response is always stored as a file rather than printed inline. 3. Both join `RUN_COMPANION_MODULE_BASENAMES`, the mechanism `composio run` already uses for its helpers, so build, release packaging, install and upgrade verification, and the self-repair download pick them up unchanged. The three hand-maintained uninstall lists and the upgrade E2E fixture gain the two file names. 4. A companion bundles its own copy of `effect`, and a fiber cannot run primitives from another copy. So nothing Effect-shaped crosses the boundary. The generation companion exposes plain promises and returns failures as values. `src/generation/errors.ts` rebuilds them as the CLI's own error classes with fields and stack intact. 5. `src/constants.ts` imported `constants` from `@composio/core`'s root entry for two strings and two URLs, which evaluated the whole SDK at startup (~25ms, mostly zod schemas). The values are inlined and a test pins them to core's. `tool-file-uploads.ts` imports its three core helpers on the upload path instead of at module scope. 6. Build guard. After building the companions, the build bundles `src/bin.ts` once more with the release build's `DEBUG_OVERRIDE_*` env inlining, defines, `NODE_ENV=production` and syntax minification (whitespace is kept, so the per-module path comments it reads survive), and fails if the executable's graph reaches `typescript`, `js-tiktoken`, `src/generation/*` or a companion entry. Checked that it fires on a stray static import. `@composio/core`'s root entry is not on the list: it is still bundled behind the file-upload path's dynamic import (see Additional context), so the guard cannot exclude it. `test/src/commands/startup-imports.test.ts` forbids the same modules when the command tree loads from source. What changes for users: - A damaged install (companion file missing) now affects `generate` the way it already affected `run`: self-repair from the release archive, then an error. A large `execute` also attempts the repair, and if that fails it stores the response with a byte-based token estimate rather than failing. Responses under 10KB never touch the encoder. `--version` and everything else are unaffected. - `composio upgrade` from a binary older than this PR copies only the companion files that binary knows about. The first `generate`, `run` or large `execute` on the new version then restores the two new files through the self-repair download. - `execute` responses over 10KB pay ~20ms more after `execute.tool_call.end` (351 to 374ms), the on-demand parse of the 2.2MB encoder file. Under 10KB, unchanged. - Errors from generation are rebuilt instances. Same class, tag, fields, message and stack; different object identity. Generated output is byte-identical to #4468 for `generate ts`, `generate ts --transpiled` and `generate py`. The 11-invocation help/error diff from #4468 is identical. Found on the way: `assertBundledRuntimeFiles` blanked string literals to same-length runs of spaces, and the import patterns' `^\s*` then backtracked quadratically over the compiler's embedded lib strings. The build hung for over ten minutes. String bodies are dropped now. The check has also never matched a specifier, since the specifiers it looks for are the strings it removes. Left as is, because a corrected version flags false positives in `run-subagent-output-mcp`. ## Type of change - [ ] Bug fix - [ ] New feature - [x] Refactor/Chore - [ ] Documentation - [ ] Breaking change ## How Has This Been Tested? Bun 1.4.1+4661e494f, Node 24.20.0, pnpm 11.8.0, linux-x64. 1. `cd ts/packages/cli && pnpm run typecheck && pnpm run validate:boundaries && pnpm run validate:skills` 2. `pnpm exec vitest run`: 129 files, 1335 passed, 1 skipped. New tests cover the mirrored constants, error rehydration and outcome lifting, and the loader resolving both companions from source. 3. `pnpm build:binary`, then against `dist/composio`: `generate ts`, `generate ts --transpiled` and `generate py` diffed against #4468's binary, `run` with a trailing expression, `execute` with 1.6KB and 35KB responses, and the damaged-install cases with files deleted from `dist/`. 4. Docker E2E on this branch: `upgrade` 2 pass, `run` 8 pass, `version` 9 pass, `install` 7 pass on bash and 5 pass on zsh. The install runs used a fixture built the way CI builds it (`build:binary:cross`, `build:binary:package`, `build:binary:checksums`), which also confirms the release zip carries both new files. 5. `bun run test/release-workflow.test.ts` at the repo root, for the synced uninstall lists. 6. Follow-up commit (encoder fallback, typed load failure, graph-check and tsdown fixes): `pnpm run typecheck`, `validate:boundaries` and oxlint pass. The execute, companion-loader, constants, generation-runtime, `run` and `generate` suites pass (177 passed, 1 skipped), including new tests for the estimate when the encoder cannot load and for the typed load failure. The fallback test fails without the fix. `pnpm build` emits both companions, and `pnpm build:binary` passes the graph check. 7. Review follow-ups (companion repair scoped to the requested module and run before the import, required-export check, stored output when the token count is an estimate, graph check using the release build inputs, startup-imports list): `pnpm run typecheck`, prettier and oxlint pass. Full `pnpm exec vitest run` on the CLI package: 1340 passed, 1 skipped, 1 failure in `analytics.dispatch.test.ts`, which this PR does not touch and which fails 1 run in 3 on its own. After the last loader change, the companion-loader, execute, `run`, `generate`, generation-runtime, startup-imports and upgrade suites pass (198 passed, 1 skipped). The new fallback test for a response whose estimate is under the threshold fails without its fix. `bun run ./scripts/build-companion-modules.ts` passes the updated graph check. ## Screenshots (if applicable) Not applicable. ## Checklist - [x] I have read the Code of Conduct and this PR adheres to it - [x] I ran linters/tests locally and they passed - [x] I updated documentation as needed - [x] I added tests or explain why not applicable - [ ] I added a changeset if this change affects published packages `@composio/cli` is private, so no changeset. Docs: the uninstall lists and the code generation section of `ts/packages/cli/AGENTS.md`. ## Additional context `openai` and `pusher-js` (~0.5MB minified) are still in the executable. Only `@composio/core`'s root entry reaches them, and the two upload guards have no lighter subpath export. A `@composio/core/utils/file-upload-guard` entry would remove them; that is a core package change. Companion files carry no version stamp. The loader checks that a companion has every export its caller uses, so a file from another release missing one fails with a reinstall error. A file from another version with the same exports still loads as is; checking `APP_VERSION` after import would catch that, and it is not done here. The remaining ~95ms of module evaluation is a long tail of eager Schema and command definitions across `src/commands`, `src/services`, `effect` and `src/models`, not one dependency.

Daksh committed Sep 14, 2026 at 20:32 UTC 9b1cbb180b9d7221bff468e55ce598bcb25d7860

No changes.