Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.
Mirrored from https://github.com/Camo67/composio.git
Synced Nov 21, 07:15 UTC
| 1 | # Verification Map |
| 2 | |
| 3 | The maintained argv and working directories are in `scripts/agent-profiles.json`. |
| 4 | Run from the repository root with its pinned mise toolchain, Bun and the normal |
| 5 | or Python development environment prepared. Profiles run existing checks and |
| 6 | their normal environment setup (including nox environment creation); they do not |
| 7 | choose a model. |
| 8 | |
| 9 | The CLI and TypeScript SDK profiles use package-scoped Turbo tasks so prerequisite |
| 10 | workspace builds run before tests and type checks, including in a fresh checkout. |
| 11 | |
| 12 | | Profile | Existing checks | What remains outside its proof | |
| 13 | | ------------------ | ------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | |
| 14 | | `workflow` | Bun task integration tests, their TypeScript check and both agent-guidance validators | Agent judgment, product behavior, hosted CI | |
| 15 | | `cli` | CLI package tests, boundary validators and TypeScript checks | Built-binary behavior, real backend behavior | |
| 16 | | `sdk-typescript` | Core SDK unit tests and type checks | Changed providers, runtime packaging, live API behavior | |
| 17 | | `sdk-python` | Existing `make chk` and `make tst` nox sessions in `python/` | Built artifacts, live API behavior, providers outside those sessions | |
| 18 | | `release-metadata` | Existing release-workflow checks | Hosted checks, merges, tags, registries and clean consumer installation | |
| 19 | |
| 20 | Use the package skills for narrower tests or additional affected providers. For |
| 21 | packaging or runtime changes, use existing Node/Cloudflare or CLI Docker E2E |
| 22 | checks after reading `typescript-testing` or `cli-e2e`. For publication, follow |
| 23 | `cli-release` or `python-release`; a local receipt never establishes publication. |
| 24 | |
| 25 | ## Carry a task between sessions |
| 26 | |
| 27 | The TypeScript CLI runs with Bun and uses `effect/cli` for argument validation and |
| 28 | generated help. Start with `bun run agent:task --help` or |
| 29 | `bun run agent:task init --help` to inspect commands and required flags. |
| 30 | Task commands print JSON; verification streams check output to stderr. |
| 31 | |
| 32 | ```sh |
| 33 | bun run agent:task init cli-output --goal "Reproduce the reported CLI output bug and verify the fix" --profile cli --source https://github.com/ComposioHQ/composio/issues/123 --authority "Implement locally; publication requires the current request" |
| 34 | bun run agent:task status cli-output |
| 35 | bun run agent:task verify cli-output |
| 36 | bun run agent:task list |
| 37 | ``` |
| 38 | |
| 39 | The id names `.artifacts/agent-tasks/<id>/task.json`. Update its `nextStep` and |
| 40 | `blocker` fields as work progresses. Keep the goal observable; record the real |
| 41 | source issue or PR URL when one exists. The authority field preserves context; |
| 42 | it is never permission to exceed the current user request or repository rules. |
| 43 | |
| 44 | Receipts retain the task, profile, source-content fingerprint and exact HEAD at |
| 45 | verification time. A changed task acceptance contract, tracked or untracked source file, or profile |
| 46 | makes previous passing evidence stale. Ignored artifacts and Claude Code's |
| 47 | `.claude/scheduled_tasks.lock` are excluded, so a |
| 48 | receipt does not cover ignored binaries or environment state changing later. |
| 49 | Fresh external observations are still required for production and registry claims. |
| 50 | Changing only `nextStep` or `blocker` preserves verification evidence. |
| 51 | |
| 52 | Each attempt has its own receipt and a private output log capped at 64 KiB. Longer |
| 53 | output keeps its first 16 KiB and last 48 KiB around an omission marker, so final |
| 54 | errors survive. The log is saved while commands run, so it keeps recent output when the |
| 55 | runner is killed. Attempts are numbered in order, not by clock time. Failed attempts survive subsequent success. Missing tools, failed commands, interruption |
| 56 | and source edits during verification cannot produce a passing attempt. Remaining |
| 57 | commands after the first failure were not run. An abruptly killed runner reads as |
| 58 | interrupted, including when its latest attempt has no finished receipt. A retry |
| 59 | recovers a dead runner's lock once its old command group is gone. If that group |
| 60 | remains alive, or the runner died before saving the command's process group, the |
| 61 | error identifies the command and receipt to inspect before retrying. Live-runner |
| 62 | locks are never recovered automatically. An empty or corrupt `verify.lock` is |
| 63 | replaced only when no task has a running attempt whose runner or command group is |
| 64 | alive. A command that exits while workers it started are still running does not end |
| 65 | its step: the runner stops those workers before the next command or lock release. |
| 66 | |
| 67 | Verification is serialized per worktree because profiles share build and nox outputs. |
| 68 | Use separate worktrees for parallel tasks. An interrupted lock acquisition can leave |
| 69 | `.artifacts/agent-locks/acquire.lock`; inspect its PID and remove it only after |
| 70 | confirming that process has ended. |
| 71 | |
| 72 | ## Learn from interventions |
| 73 | |
| 74 | `test/agent-task.test.ts` turns concrete workflow failure classes into runnable |
| 75 | checks: context survives resumption; edits invalidate proof; failures preserve |
| 76 | evidence; missing tools fail rather than skip. These exercise the tool's behavior, |
| 77 | not an LLM's planning quality. For a new recurring mistake, add the smallest |
| 78 | behavioral check at the real failure boundary and demonstrate its failure before |
| 79 | the fix. Keep unrelated passing cases as retention checks. Assess judgment-heavy |
| 80 | changes on new tasks before claiming they reduce supervision. |