Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.
| 1 | # Batch 2026-11-11T18-56-18-claude-code-claude-haiku-5-5-link-url-off-terminal |
| 2 | |
| 3 | claude-code 2.1.310, claude-haiku-5-5, 5 run(s) per journey and arm. |
| 4 | Revision 0f1477f51; skill 0cc1b6f91c3b; harness 9ff7364caa93. |
| 5 | Spent $0.033 on new runs; 0 run(s) reused. |
| 6 | |
| 7 | | split | journey | arm | passed | pass@1 (95% CI) | pass@k | pass^k | wrong tree | mean cost | infra | |
| 8 | | ----- | ----------------------------- | ----- | ------ | --------------- | ------ | ------ | ---------- | --------- | ----- | |
| 9 | | train | consumer-slack-message | skill | 5/5 | 100% (57%–100%) | 100% | 100% | 0% | $0.003 | 0 | |
| 10 | | train | developer-execute-unconnected | skill | 5/5 | 100% (57%–100%) | 100% | 100% | 0% | $0.003 | 0 | |
| 11 | | train | first-run-login | skill | 3/5 | 60% (23%–88%) | 100% | 0% | 0% | $0.001 | 0 | |
| 12 | |
| 13 | | split | arm | mean pass@1 | mean pass^k | mean wrong tree | |
| 14 | | ----- | ----- | ----------- | ----------- | --------------- | |
| 15 | | train | skill | 87% | 67% | 0% | |
| 16 | |
| 17 | ## Failed checks |
| 18 | |
| 19 | - first-run-login (skill): |
| 20 | - 2/5 the model finished by itself, within the turn limit |
| 21 | - 2/5 exactly one CLI login session was created, and it was linked |
| 22 | - 2/5 the login completed: the user API key was fetched for the linked session |
| 23 | - 2/5 the login was stored: a new `composio whoami` exits 0 as the person, in their org |