@devries-cameron20 / composio Mirror · read-only / commits / feca0389f
Fork 0

Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

Mirrored from https://github.com/Camo67/composio.git

Synced Nov 21, 12:15 UTC

feat(cli): plugin-adoption telemetry for setup and the plugin hint (#4496)

## Summary Live PostHog (2026-09-15): since auto-setup was restored on Sep 14, 704 installer-triggered `composio setup` runs found no host 397 times (56%); of the 316 that found one, 282 installed the plugin (89%). The gap is host detection, and today a real absence is indistinguishable from a PATH miss. Manual `composio setup` over 30 days: 888 succeeded, 626 failed, and `CLI_SETUP_FAILED` only carried `error_name`. 103 of those failures (`yes=false, target=auto, stdout_is_tty=false`) are agents following the daily hint text into "Non-interactive setup requires `--yes`". No event said whether the CLI was running inside Claude Code or Codex at all. Event contract (metrics.composio.io is being built against these names): - Every CLI event gains `agent_host_env: 'claude' | 'codex' | 'none'`, derived from `CLAUDECODE` / `CODEX_THREAD_ID` / `CODEX_SANDBOX`. - `CLI_SETUP_HOST_DETECTED` gains `host_config_dir_present` and `host_binary_in_known_paths` when `available=false` (`$CLAUDE_CONFIG_DIR`/`~/.claude`, `$CODEX_HOME`/`~/.codex`; `~/.claude/local`, `~/.local/bin`, `~/.npm-global/bin`, `/usr/local/bin`, `/opt/homebrew/bin`). Both omitted when the host is detected. - New `CLI_PLUGIN_HINT_SHOWN` (journey stage `setup`) with `source`, `invocation_origin`, `cli_version`, `command_path`, `agent_host`, emitted once per printed hint and never on suppression. - `CLI_SETUP_FAILED` gains `failure_reason_code`: `all_requires_both_hosts | unsupported_host | target_not_installed | no_host_detected | non_interactive_requires_yes | marketplace_conflict | unknown`, carried on `SetupCommandError.reasonCode`. - Hint text is now `Tip: running under <host> without the Composio plugin — 'composio setup --yes' installs it.` Structural notes: `agent_host_env` is stamped in `trackCliEventEffect` (`analytics/dispatch.ts`) next to `org_id`, so every enqueued envelope carries it with no module state or bootstrap hook. `SetupCommandError` and `SetupFailureReasonCode` live in the leaf module `services/setup-command-error.ts` (imports only `effect`) so `analytics/events.ts` can use `instanceof` without a cycle; `setup.cmd.ts` keeps its original `setupCommandError` helper with the reason code as a third argument, and the two validate-stage failures in `services/setup.ts` (`marketplace_conflict`, `target_not_installed`) are raised as `SetupCommandError` directly. The raw host-env read, `detectPluginHost`, and the known-path install probe live in `services/agent-host-env.ts`, shared by `dispatch.ts`, `plugin-hint.ts`, and `setup.ts`. `CLI_SETUP_HOST_DETECTED` passes the two presence booleans straight through; `setup.ts` only probes an undetected host. The hint tracks `CLI_PLUGIN_HINT_SHOWN` right where it prints. ## Validation - `pnpm --filter @composio/cli` `pnpm run test` (validate:skills, validate:boundaries, vitest): 132 files, 1374 passed, 1 skipped. - `pnpm run typecheck` (src + test): clean. - `oxlint` on the 14 changed TS files: clean. `prettier --check` on changed files: clean. `git diff --check`: clean. - `pnpm validate:agent-skills` and `pnpm validate:skill-routing`: pass (skill reference doc changed). - Manual, built binary with isolated `HOME`/`COMPOSIO_CACHE_DIR`/`CLAUDE_CONFIG_DIR` and a dummy PostHog key pointed at a dead local port: `CLAUDECODE=1 composio whoami --telemetry-debug` printed the new hint once and enqueued `CLI_PLUGIN_HINT_SHOWN` (`command_path: whoami`, `agent_host: claude`) plus `CLI_COMMAND_INVOKED`/`SUCCEEDED`, all with `agent_host_env: claude`; a second run printed no hint and no hint event. `PATH=/usr/bin:/bin composio setup --target codex --yes --telemetry-debug` enqueued `CLI_SETUP_HOST_DETECTED` with `available: false, host_config_dir_present: true, host_binary_in_known_paths: false` and `CLI_SETUP_FAILED` with `failure_reason_code: target_not_installed`. Re-run after each simplification pass with identical output; with the host markers unset the same events carry `agent_host_env: none`. ## Known verification limitations - No changeset: `@composio/cli` is ignored by Changesets per `ts/AGENTS.md`; the note went into `ts/packages/cli/CHANGELOG.md` instead. - Docker CLI E2E not run; no binary output contract changed except the hint line. - `host_binary_in_known_paths` checks two absolute directories, so the "absent everywhere" test asserts a boolean rather than `false` to stay machine-independent. Not included: any change to `composio setup` help text or to the dashboard side. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

Alberto Schiabel committed Sep 16, 2026 at 22:23 UTC feca0389f9abb79bb4ed47f89ea6cbc7f6ed40f0

No changes.