Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

next
md 36 lines 2.11 KB
Rendered Raw
1 # Batch 2026-11-11T18-14-14-claude-code-claude-haiku-5-5-dev-playground-execute
2
3 claude-code 2.1.310, claude-haiku-5-5, 5 run(s) per journey and arm.
4 Revision e4e5d922e with uncommitted changes; skill f6f4d7820d90; harness 9ff7364caa93.
5 Spent $0.039 on new runs; 0 run(s) reused.
6
7 | split | journey | arm | passed | pass@1 (95% CI) | pass@k | pass^k | wrong tree | mean cost | infra |
8 | ----- | ----------------------------- | ----- | ------ | --------------- | ------ | ------ | ---------- | --------- | ----- |
9 | train | consumer-slack-message | skill | 4/5 | 80% (38%–96%) | 100% | 0% | 0% | $0.003 | 0 |
10 | train | developer-execute-unconnected | skill | 0/5 | 0% (0%–43%) | 0% | 0% | 0% | $0.003 | 0 |
11 | train | first-run-login | skill | 4/5 | 80% (38%–96%) | 100% | 0% | 0% | $0.001 | 0 |
12
13 | split | arm | mean pass@1 | mean pass^k | mean wrong tree |
14 | ----- | ----- | ----------- | ----------- | --------------- |
15 | train | skill | 53% | 0% | 0% |
16
17 ## Failed checks
18
19 - consumer-slack-message (skill):
20 - 1/5 SLACK_SEND_MESSAGE completed once in the whole run
21 - 1/5 it ran on the consumer project, as consumer-user-e2e-acme-0001
22 - 1/5 it went to #general
23 - 1/5 its text was exactly "deploy finished"
24 - 1/5 the account was linked on the consumer project
25 - developer-execute-unconnected (skill):
26 - 5/5 SLACK_SEND_MESSAGE completed once in the whole run
27 - 5/5 it ran on the developer project, as pg-test-00000000-0000-4000-8000-00000000a001
28 - 5/5 it went to #general
29 - 5/5 its text was exactly "hello from dev"
30 - 5/5 the account was linked on the developer project for the test user
31 - 5/5 the last `composio dev playground-execute` exited 0
32 - first-run-login (skill):
33 - 1/5 the model finished by itself, within the turn limit
34 - 1/5 exactly one CLI login session was created, and it was linked
35 - 1/5 the login completed: the user API key was fetched for the linked session
36 - 1/5 the login was stored: a new `composio whoami` exits 0 as the person, in their org