@smbcloud / cli
Code Commits Issues Pulls Cloud Agent CI/CD
main
claude/smbcloud-github-app-k9641p development feature/apps feature/build-release feature/ci-docs feature/ci-ruby-gems feature/cicd feature/completion feature/dist-nuget feature/fix-ci-sdk-pypi-rubygem feature/fix-clippy-on-stable-channel feature/fix-deployment-script feature/github-email feature/gresiq-v1 feature/project-new-structure feature/rsync-deploy feature/ruby-runner feature/sdk-npm feature/sdk-workflows feature/smbcloud-cli-release-npm feature/smbcloud-deploy feature/wasm main release/v0.7.4
md 164 lines 6.39 KB
Rendered Raw
1 # ControlKit runners
2
3 ControlKit is the native XCTest runner used by the CLI's Apple-platform MCP
4 tools. The public runner project is
5 [`xcrs-controlkit`](https://github.com/ondeinference/xcrs-controlkit). It uses
6 one Xcode project generated by XcodeGen, with platform-specific targets and
7 schemes:
8
9 | Platform | Scheme | App product |
10 | --- | --- | --- |
11 | iOS / iPadOS | `ControlKit-iOS` | `ControlKit.app` |
12 | tvOS | `ControlKit-tvOS` | `ControlKit.app` |
13 | visionOS | `ControlKit-visionOS` | `ControlKit.app` |
14 | watchOS | `ControlKit-watchOS` | `ControlKit.app` |
15 | macOS | `ControlKit-macOS` | `ControlKit.app` |
16
17 The scheme and target names keep their platform suffixes so Xcode can select
18 the correct SDK, while each built app is named `ControlKit`.
19
20 ## Build a runner
21
22 Install XcodeGen once, clone the runner project, and generate the Xcode
23 project:
24
25 ```sh
26 brew install xcodegen
27 cd <path-to-xcrs-controlkit>/Runners
28 xcodegen generate
29 ```
30
31 Build the simulator runner for the platform you need:
32
33 ```sh
34 xcodebuild -project XCRSControlKitRunner.xcodeproj \
35 -scheme ControlKit-<platform> \
36 -destination 'generic/platform=<platform> Simulator' \
37 build-for-testing
38 ```
39
40 Replace `<platform>` with `iOS`, `tvOS`, `watchOS`, or `visionOS`. macOS is
41 built with `-destination 'platform=macOS'` and runs on the local Mac rather
42 than a simulator.
43
44 ## Start the runner
45
46 Boot a compatible simulator and run the UI-test host:
47
48 ```sh
49 xcodebuild -project XCRSControlKitRunner.xcodeproj \
50 -scheme ControlKit-<platform> \
51 -destination 'id=<simulator-udid>' \
52 test-without-building
53 ```
54
55 The shared XCTest runner starts a local ControlKit JSON-RPC server on
56 `127.0.0.1:12004`. Keep that test process running while calling the MCP tools.
57 The macOS UI-test target includes the network-server entitlement required to
58 bind the local port.
59
60 ## Run on a physical device
61
62 The same runner also works installed on a real iOS, tvOS, watchOS, or
63 visionOS device, not just a simulator. Build and start it against the
64 device's UDID (find it with `xcrun devicectl list devices`) instead of a
65 simulator destination:
66
67 ```sh
68 xcodebuild -project XCRSControlKitRunner.xcodeproj \
69 -scheme ControlKit-<platform> \
70 -destination 'id=<device-udid>' \
71 build-for-testing
72
73 TEST_RUNNER_CONTROLKIT_LISTEN_HOST=0.0.0.0 xcodebuild \
74 -project XCRSControlKitRunner.xcodeproj \
75 -scheme ControlKit-<platform> \
76 -destination 'id=<device-udid>' \
77 test-without-building
78 ```
79
80 The `CONTROLKIT_LISTEN_HOST` environment variable controls what the
81 in-process RPC server binds to (`Shared/ControlKitRPCServer.swift`) — it
82 defaults to `127.0.0.1`, which is loopback-only *on the device itself* and
83 unreachable from your Mac. `xcodebuild` forwards any `TEST_RUNNER_`-prefixed
84 environment variable into the test process with the prefix stripped, so
85 setting `TEST_RUNNER_CONTROLKIT_LISTEN_HOST=0.0.0.0` makes the server bind on
86 every interface, including the one reachable over your LAN. Find the device's
87 address with its Bonjour hostname (`<device-name>.local`, discoverable via
88 `dns-sd -B _airplay._tcp local` for an Apple TV) or its CoreDevice tunnel
89 address (`xcrun devicectl device info details --device <udid>`, under
90 `Tunnel IP Address`).
91
92 ## Connect through the automation MCP profile
93
94 Start either automation server:
95
96 ```sh
97 xcrs --mcp
98 # or
99 smb --mcp --scope automation
100 ```
101
102 The runner tools accept either `simulator_name` or `simulator_udid` for a
103 simulator, or `host` (plus `controlkit_port`, defaulting to `12004`) for a
104 physical device or any other remote runner. Call `device_select` once to
105 remember the target so later tools need no target arguments. Both commands
106 expose the same unprefixed tool names.
107
108 | Tool | Runner | Purpose |
109 | --- | --- | --- |
110 | `device_select` | All platforms | Remember the active simulator or device for later tools. |
111 | `device_capabilities` | All platforms | Read platform and capability information. |
112 | `input_click` | macOS | Click at screen coordinates. |
113 | `input_spatial_tap` | visionOS | Perform a spatial tap. |
114 | `input_tap` | iOS / tvOS / watchOS, simulator or physical | Perform a touch tap. |
115 | `input_text` | iOS / tvOS, simulator or physical | Type into the focused field. |
116 | `input_swipe` | iOS / tvOS, simulator or physical | Swipe between coordinates. |
117 | `input_button` | iOS / tvOS, simulator or physical | Press a Home or tvOS remote button. |
118 | `app_launch`/`app_terminate` | simulator or physical | Launch/terminate an app by bundle ID. On a physical device this calls the runner's `device.apps.launch`/`device.apps.terminate` RPC methods instead of `simctl`. |
119 | `screen_capture` | Simulator or physical | Capture a PNG screenshot. Uses `simctl` for a simulator, or `devicectl device capture screenshot` when a `device` identifier is given. |
120 | `ui_describe` | All UI-test runners | Read the accessibility hierarchy for the supplied `bundle_id`. |
121 | `ui_element_list` | All UI-test runners | Read actionable elements with tap coordinates for the supplied `bundle_id`. |
122 | `ui_tap` | All UI-test runners | Resolve and activate an accessibility element by label, identifier, or value. |
123
124 UI introspection requires a recent ControlKit runner that implements
125 `device.dump.ui`. The `ui_tap` action requires a runner that implements
126 `device.ui.tap`; it resolves and activates the requested element inside the
127 target app, so an MCP client does not need to turn accessibility coordinates
128 into a second action. The label, identifier, or value must match exactly one
129 element. On tvOS, that element must already have focus; use `input_button` to
130 navigate focus before calling `ui_tap`. All three UI tools require the target
131 app's `bundle_id`; this lets XCTest attach to an already-running app instead of
132 accidentally reading the ControlKit host app.
133
134 For a local macOS runner, omit simulator fields and pass `host` and
135 `controlkit_port` when needed:
136
137 ```json
138 {
139 "host": "127.0.0.1",
140 "controlkit_port": 12004,
141 "x": 400,
142 "y": 300
143 }
144 ```
145
146 For a physical device, pass its Bonjour hostname or tunnel address as `host`
147 instead:
148
149 ```json
150 {
151 "host": "<device-name>.local",
152 "controlkit_port": 12004,
153 "button": "select"
154 }
155 ```
156
157 Platform-specific operations intentionally reject unsupported input. For
158 example, visionOS rejects raw touch, watchOS rejects pointer and spatial input,
159 and macOS rejects touch and Home-button requests.
160
161 ## See also
162
163 For the product-level overview of end-to-end testing with xcrs and
164 ControlKit, see [smbcloud.xyz/xcrs-end-to-end-testing](https://smbcloud.xyz/xcrs-end-to-end-testing).