Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.
| 1 | # Skill decisions |
| 2 | |
| 3 | One directory per decision about the `composio-cli` skill or the CLI text agents read: why the change was made, what was measured, and the batches it was measured with. `results/` is git-ignored; this directory is where the evidence behind a kept or dropped change stays. |
| 4 | |
| 5 | A baseline has a directory of its own, named `baseline-*`, when the harness changed and the older batches can no longer be compared. It holds no decision: it is the line later decisions are measured from. See [`baseline-grader-guards`](./baseline-grader-guards/README.md). |
| 6 | |
| 7 | Each decision has: |
| 8 | |
| 9 | - `README.md`: the problem seen in the runs, the change, the commands that measured it, the comparison, and the decision. |
| 10 | - `<batch id>/`: `batch.json`, `runs.jsonl` and `summary.md` of every batch the decision rests on, copied with `bun run eval.ts keep <batch> <decision>`. The full logs are not kept; rerun the batch to get them. |
| 11 | |
| 12 | To check a decision again, compare the kept batches, then run the same commands on the current checkout and compare the new batch with the kept baseline: |
| 13 | |
| 14 | ```sh |
| 15 | bun run eval.ts compare evals/<decision>/<baseline id> evals/<decision>/<candidate id> |
| 16 | bun run eval.ts run --model claude-haiku-5-5 --runs 5 --label recheck |
| 17 | bun run eval.ts compare evals/<decision>/<candidate id> <new batch id> |
| 18 | ``` |
| 19 | |
| 20 | `compare` refuses batches graded by a different harness. When the journeys or the checks changed since, measure a new baseline at the decision's revision instead. |