1
+---
2
+description: Full CLI test pipeline — monitor CI for types/lint, then run local binary test, then run bundled binary test via CI.
3
+---
4
+
5
+# Full CLI Test
6
+
7
+End-to-end CLI validation pipeline that ensures types/lint pass in CI, then tests the CLI binary built from source, and finally tests the CI-bundled binary.
8
+
9
+## Overview
10
+
11
+This skill runs three phases sequentially:
12
+
13
+1. **Phase 1: CI Lint/Type Check** — Poll CI until the CLI type-check and lint jobs pass.
14
+2. **Phase 2: Local Binary Test** — Build and test the CLI binary from source (`/cli-test`).
15
+3. **Phase 3: Bundled Binary Test** — Trigger CI binary build, download, and test (`/cli-test-with-bundling`).
16
+
17
+## Phase 1: Monitor CI for Types/Lint
18
+
19
+Use `/loop 5m` to poll the CI status for the current branch every 5 minutes. Check that **all type-check and lint jobs pass** for the CLI package before proceeding.
20
+
21
+### How to check
22
+
23
+```bash
24
+# Get the latest CI run for the current branch
25
+gh run list \
26
+ --repo ComposioHQ/composio \
27
+ --branch "$(git rev-parse --abbrev-ref HEAD)" \
28
+ --limit 5 \
29
+ --json databaseId,status,conclusion,name,headBranch
30
+```
31
+
32
+Look for workflow runs that cover type-checking and linting (e.g., the main CI workflow). If none are running or completed yet, wait for one to appear.
33
+
34
+### What to verify
35
+
36
+- All type-check jobs: **pass**
37
+- All lint jobs: **pass**
38
+- If any fail, report the failure to the user and stop — do NOT proceed to Phase 2.
39
+
40
+Use `/loop 5m` to poll: run the `gh run list` / `gh run view` command every 5 minutes until the relevant jobs succeed. Once all type/lint checks are green, move on.
41
+
42
+## Phase 2: Local Binary Test (`/cli-test`)
43
+
44
+Once CI lint/types pass, run the `/cli-test` skill:
45
+
46
+1. Install dependencies: `pnpm install`
47
+2. Build all packages: `pnpm turbo build`
48
+3. Build the standalone binary: `pnpm --dir ts/packages/cli build:binary`
49
+4. Test the binary:
50
+ ```bash
51
+ ./ts/packages/cli/dist/composio version
52
+ ./ts/packages/cli/dist/composio whoami
53
+ ./ts/packages/cli/dist/composio --help
54
+ ```
55
+5. Run the Slack integration test (see [Slack Integration Test](#slack-integration-test) below):
56
+ ```bash
57
+ ./ts/packages/cli/dist/composio run '<SLACK_TEST_SCRIPT>'
58
+ ```
59
+
60
+If any command fails, report to the user and stop — do NOT proceed to Phase 3.
61
+
62
+## Phase 3: Bundled Binary Test (`/cli-test-with-bundling`)
63
+
64
+Once the local binary test passes, run the `/cli-test-with-bundling` skill:
65
+
66
+1. Read the version from `ts/packages/cli/package.json` and append a `-beta.<timestamp>` suffix (e.g. `1.2.3-beta.20260331143022`) — **always trigger as a beta release**
67
+2. Trigger the `build-cli-binaries.yml` workflow via `gh workflow run` with the beta version
68
+3. Monitor the workflow run (use `/loop 5m` to poll)
69
+4. Download the platform-appropriate binary artifact
70
+5. Test the binary:
71
+ ```bash
72
+ $BINARY version
73
+ $BINARY whoami
74
+ $BINARY --help
75
+ $BINARY run 'console.log("hello from composio run")'
76
+ $BINARY run 'const result = await experimental_subAgent({ goal: "What is 2+2?", toolNames: [] }); console.log(result)'
77
+ ```
78
+6. Run the Slack integration test (see [Slack Integration Test](#slack-integration-test) below):
79
+ ```bash
80
+ $BINARY run '<SLACK_TEST_SCRIPT>'
81
+ ```
82
+7. If on a PR, post results as a PR comment
83
+
84
+## Slack Integration Test
85
+
86
+This test validates `execute()`, `experimental_subAgent()`, and end-to-end Slack connectivity. It must be run in **both Phase 2 and Phase 3** to ensure it works with both the locally-built and CI-bundled binaries.
87
+
88
+### Channel
89
+
90
+**`#buzz-skill-based-cli-testing`** — dedicated Slack channel for automated CLI test runs.
91
+
92
+### Test Script
93
+
94
+Use `composio run` (or `$BINARY run` in Phase 3) with the following script. Replace `$BINARY` with the appropriate binary path for the phase.
95
+
96
+```bash
97
+$BINARY run '
98
+ // Step 1: Find the #buzz-skill-based-cli-testing channel
99
+ const channels = await execute("SLACK_LIST_CHANNELS", {
100
+ types: "public_channel",
101
+ limit: 200,
102
+ });
103
+ const channel = channels.data?.channels?.find(
104
+ (c) => c.name === "buzz-skill-based-cli-testing"
105
+ );
106
+ if (!channel) throw new Error("Channel #buzz-skill-based-cli-testing not found");
107
+
108
+ // Step 2: Send an initial message tagging @cryogenicplanet
109
+ const buildType = "local"; // use "bundled" in Phase 3
110
+ await execute("SLACK_SEND_A_MESSAGE_TO_A_SLACK_CHANNEL", {
111
+ channel: channel.id,
112
+ text: `<@cryogenicplanet> CLI test run started (${buildType} build) at ${new Date().toISOString()}`,
113
+ });
114
+
115
+ // Step 3: Fetch recent channel history for the subAgent to summarize
116
+ const history = await execute("SLACK_GET_CHANNEL_HISTORY", {
117
+ channel: channel.id,
118
+ limit: 20,
119
+ });
120
+
121
+ // Step 4: Use experimental_subAgent to summarize what happened in the channel
122
+ const summary = await experimental_subAgent(
123
+ `Summarize the recent activity in this Slack channel in 2-3 sentences. Focus on what tests were run and their outcomes.\n\n${history.prompt()}`,
124
+ {
125
+ schema: z.object({
126
+ summary: z.string(),
127
+ messageCount: z.number(),
128
+ }),
129
+ }
130
+ );
131
+
132
+ // Step 5: Post the summary back to the channel
133
+ await execute("SLACK_SEND_A_MESSAGE_TO_A_SLACK_CHANNEL", {
134
+ channel: channel.id,
135
+ text: `CLI Test Summary (${buildType} build):\n${summary.structuredOutput.summary}\n(${summary.structuredOutput.messageCount} messages analyzed)`,
136
+ });
137
+
138
+ console.log("Slack integration test passed:", summary.structuredOutput);
139
+'
140
+```
141
+
142
+### What this validates
143
+
144
+| Capability | How it's tested |
145
+|---|---|
146
+| `execute()` with Slack tools | List channels, send messages, fetch history |
147
+| `experimental_subAgent()` | Summarizes channel history with structured output via `z` schema |
148
+| `z` (Zod) global | Used in the subAgent schema definition |
149
+| `result.prompt()` | Feeds channel history into the subAgent |
150
+| End-to-end Slack connectivity | Reads from and writes to a real Slack channel |
151
+
152
+### Important Notes
153
+
154
+- Change `buildType` to `"bundled"` when running in Phase 3.
155
+- The Slack user tag `<@cryogenicplanet>` should resolve to the correct user in your workspace. If the mention doesn't resolve, find the Slack user ID first and use `<@U_XXXXX>` format.
156
+- This test requires an active Slack connection in Composio. If not connected, run `composio connect slack` first.
157
+
158
+## Failure Handling
159
+
160
+- **Phase 1 failure (lint/types):** Report which jobs failed with links. Do not continue.
161
+- **Phase 2 failure (local build/test):** Report the error output. Do not continue.
162
+- **Phase 3 failure (bundled build/test):** Report which commands failed. Post results to PR if applicable.
163
+
164
+## Reference Files
165
+
166
+| File | Purpose |
167
+|---|---|
168
+| `.github/workflows/build-cli-binaries.yml` | CI workflow that builds binaries |
169
+| `ts/packages/cli/package.json` | Source of CLI version |
170
+| `ts/packages/cli/scripts/build-binary.ts` | Local binary build script |
171
+| `ts/packages/cli/dist/composio` | Built binary output |