@devries-cameron20 / composio Mirror · read-only
Fork 0

Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

Mirrored from https://github.com/Camo67/composio.git

Synced Nov 21, 07:15 UTC

next
md 36 lines 3.3 KB
Rendered Raw
1 # Print the authorization URL when `link` waits off a terminal
2
3 **Decision: kept.** This is a CLI fix, with one skill sentence, measured with the same journeys as the skill decisions.
4
5 ## What the runs showed
6
7 After [the developer-project skill change](../2026-11-11-dev-playground-execute/README.md), Haiku chose the right tree on `developer-execute-unconnected` in every run and still passed none of them. Each run stopped at `composio dev connected-accounts link slack --no-browser`, which printed nothing while it waited: the URL went through terminal decoration (`ui.log.step`, `ui.note`), which `TerminalUI` drops when stderr is not a terminal. The agent had no URL to give the person, nobody approved, and the wait never ended. The wait was also longer than meant: `Schedule.max` of an exponential ramp and a 5-second spacing waits for the longer of the two, so fifteen checks took about 2 hours 44 minutes, and `composio login`'s hint promised 10 minutes.
8
9 ## The change
10
11 - `connected-accounts.link.format.ts`: off a terminal, a `link` that waits writes the URL to stderr with `ui.error`, which survives redirection, followed by a hint for agents that names `--no-wait`. `--no-wait` keeps stderr quiet; its stdout JSON already carries the URL.
12 - `src/effects/authorization-poll.ts`: one schedule for `login` and `link`, a 0.3s ramp capped at one check every 5 seconds, for 10 minutes.
13 - `skills-src/composio-cli/index.ts`: the `link` section shows `--no-browser --no-wait` and says to give the user the `redirect_url` and wait for them.
14
15 ## Results
16
17 Haiku, skill arm, train, 5 runs. Baseline: the kept `…-dev-playground-execute` batch of the previous decision. Same harness (`9ff7364caa93`); the CLI binary and the skill differ.
18
19 | journey | pass@1 | pass^5 | wrong tree |
20 | ----------------------------- | ---------- | --------- | ---------- |
21 | consumer-slack-message | 80% → 100% | 0% → 100% | 0% → 0% |
22 | developer-execute-unconnected | 0% → 100% | 0% → 100% | 0% → 0% |
23 | first-run-login | 80% → 60% | 0% → 0% | 0% → 0% |
24
25 - `developer-execute-unconnected` is what the change targets. All five runs linked with `composio dev connected-accounts link slack --no-browser --no-wait`, gave the person the `redirect_url`, and sent the message after the person's reply. None of them waited on `link`, so the gain comes from the skill sentence that teaches `--no-wait`, not from the URL now printed on stderr. That one is for an agent that still runs `link` without `--no-wait`, and it is checked by the handler tests and by `operations/process-streams.test.ts`, not by these runs.
26 - The two `first-run-login` failures are the failure seen before this change, at the same rate within noise: the agent says to open the login URL without writing it, so the person cannot approve and `composio login --poll` waits out the run's deadline. `login` was not changed apart from its wait now ending after 10 minutes.
27
28 Spend: $0.03.
29
30 ## Repeat it
31
32 ```sh
33 bun run eval.ts compare evals/2026-11-11-dev-playground-execute/2026-11-11T18-14-14-claude-code-claude-haiku-5-5-dev-playground-execute \
34 evals/2026-11-11-link-url-off-terminal/2026-11-11T18-56-18-claude-code-claude-haiku-5-5-link-url-off-terminal
35 bun run eval.ts run --model claude-haiku-5-5 --runs 5 --arms skill --split train --label recheck
36 ```