Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

next
md 25 lines 1.46 KB
Rendered Raw
1 # Batch 2026-11-11T18-25-02-claude-code-claude-opus-5-5-opus-baseline
2
3 claude-code 2.1.310, claude-opus-5-5, 3 run(s) per journey and arm.
4 Revision e4e5d922e with uncommitted changes; skill 72d9d691700d; harness 9ff7364caa93.
5 Spent $0.604 on new runs; 0 run(s) reused.
6
7 | split | journey | arm | passed | pass@1 (95% CI) | pass@k | pass^k | wrong tree | mean cost | infra |
8 | ------- | ---------------------------------- | ----- | ------ | --------------- | ------ | ------ | ---------- | --------- | ----- |
9 | train | developer-execute-unconnected | skill | 0/3 | 0% (0%–56%) | 0% | 0% | 0% | $0.084 | 0 |
10 | holdout | developer-auth-configs-and-execute | skill | 3/3 | 100% (44%–100%) | 100% | 100% | 0% | $0.117 | 0 |
11
12 | split | arm | mean pass@1 | mean pass^k | mean wrong tree |
13 | ------- | ----- | ----------- | ----------- | --------------- |
14 | train | skill | 0% | 0% | 0% |
15 | holdout | skill | 100% | 100% | 0% |
16
17 ## Failed checks
18
19 - developer-execute-unconnected (skill):
20 - 3/3 SLACK_SEND_MESSAGE completed once in the whole run
21 - 3/3 it ran on the developer project, as pg-test-00000000-0000-4000-8000-00000000a001
22 - 3/3 it went to #general
23 - 3/3 its text was exactly "hello from dev"
24 - 3/3 the account was linked on the developer project for the test user
25 - 1/3 the last `composio dev playground-execute` exited 0