Split macOS computer-use backend guidance
Add a macOS-specific computer-use skill for AX structural targeting and keep the generic host skill backend-neutral. Surface backend ids, families, and advertised features from computer_use_remote start/status results, add backend-gated ax_snapshot and ax_action handling, and prompt the model to load the macOS skill only when the CLI reports matching support.
Alessandro Frau committed
May 23, 2026 at 15:08 UTC
e7cb3aa3fa42a78a63b33fbc72700e96d0376c92
7 files changed
+315
-20
plugins/_a0_connector/prompts/agent.system.tool.computer_use_remote.md
+9
-2
@@ -8,7 +8,9 @@ This is the only desktop-control path for the user's connected host/local comput
8
9
If the tool reports no CLI, disabled computer use, or `COMPUTER_USE_REARM_REQUIRED`, stop and tell the user to run `/computer-use on` in A0 CLI and approve any host permission prompt.
10
11
-Call `start_session` before screen-driven tasks. Use `status` for state only, `capture` for screenshots without an action, and `stop_session` when the desktop task is complete. Interactive actions should use normalized global-screen coordinates from the most recent capture.
11
+Call `start_session` before screen-driven tasks. Use `status` for state only, `capture` for screenshots without an action, and `stop_session` when the desktop task is complete. Interactive coordinate actions should use normalized global-screen coordinates from the most recent capture.
12
+
13
+Some actions are backend-specific. Use `ax_snapshot` and `ax_action` only when `status` or `start_session` reports a macOS backend with Accessibility-tree features such as `accessibility-tree-snapshot` or `accessibility-structural-targeting`. For macOS structural targeting details, load and follow skill `host-computer-use-macos`; do not apply AX guidance to non-macOS backends.
14
15
State-changing actions automatically attach a fresh screen after they run. Treat key presses, clicks, scrolling, typing, and window-manager shortcuts as attempts, not success: inspect the latest attached screen, or one explicit `capture` if it is unclear or unchanged, before saying the requested outcome happened. If the tool says a screen was attached but you cannot actually inspect the image, stop and report that visual verification is unavailable; do not continue by assuming the host state. For Ubuntu/GNOME/Wayland hide-window tasks, prefer `Super+H` (`{"action":"key","keys":["Super","H"]}`) for the active window; do not use `Alt+F9` as the primary hide/minimize shortcut because it often leaves the window visible. A `type` result only proves keystrokes were sent; it does not prove the window was hidden or that text landed in the intended place.
16
@@ -22,13 +24,18 @@ State-changing actions automatically attach a fresh screen after they run. Treat
24
```
25
26
Required argument:
25
-- `action`: one of `start_session`, `status`, `capture`, `move`, `click`, `scroll`, `key`, `type`, `stop_session`
27
+- `action`: one of `start_session`, `status`, `capture`, `ax_snapshot`, `ax_action`, `move`, `click`, `scroll`, `key`, `type`, `stop_session`
28
29
Optional arguments by action:
30
- `session_id`: session returned by `start_session`
31
- `x`, `y`: normalized `[0,1]` global-screen coordinates for `move` and `click`
32
- `button`: `left`, `right`, or `middle` for `click`
33
- `count`: click count for `click`
34
+- `max_depth`, `max_nodes`: optional bounds for backend-gated `ax_snapshot`
35
+- `path`: element path from `ax_snapshot` for backend-gated `ax_action`
36
+- `target`: semantic element target for backend-gated `ax_action`, for example role/title/description/value/identifier
37
+- `operation` or `ax_action`: backend-gated AX operation such as `press`, `focus`, or `set_value`
38
+- `value` or `text`: value for backend-gated `ax_action` with `set_value`
39
- `dx`, `dy`: scroll amounts for `scroll`
40
- `key` or `keys`: key press value for `key`
41
- `text`: text to type for `type`
plugins/_a0_connector/skills/host-computer-use-macos/SKILL.md
new
+83
@@ -0,0 +1,83 @@
1
+---
2
+name: host-computer-use-macos
3
+description: Backend-specific macOS guidance for `computer_use_remote`. Load after `status` or `start_session` reports backend_family/backend_id `macos` or Accessibility-tree features. Covers AX structural targeting, macOS window actions, and screenshot verification.
4
+---
5
+
6
+# Host Computer Use - macOS
7
+
8
+Use this after `host-computer-use` when the connected A0 CLI reports the macOS computer-use backend.
9
+
10
+Do not use this skill for Linux, Windows, Xpra, Docker, or browser-only tasks. If the backend is not macOS or does not advertise Accessibility-tree support, skip AX actions and follow the generic host computer-use rules.
11
+
12
+## macOS AX Targeting
13
+
14
+macOS backends can advertise structural Accessibility features:
15
+
16
+- `accessibility-tree-snapshot`
17
+- `accessibility-structural-targeting`
18
+- `accessibility-element-click`
19
+
20
+When these features are present, prefer structural targeting over pixel clicks for named controls such as buttons, menu items, text fields, sheets, alerts, toolbar items, and sidebar rows.
21
+
22
+Use `ax_snapshot` to inspect the frontmost app's bounded Accessibility tree:
23
+
24
+```json
25
+{
26
+ "tool_name": "computer_use_remote",
27
+ "tool_args": {
28
+ "action": "ax_snapshot",
29
+ "max_depth": 4,
30
+ "max_nodes": 200
31
+ }
32
+}
33
+```
34
+
35
+The snapshot returns element paths, roles, labels, frames, enabled/focused state, actions, and child nodes. Use it to choose an element, not as final visual proof.
36
+
37
+Use `ax_action` for a structural action:
38
+
39
+```json
40
+{
41
+ "tool_name": "computer_use_remote",
42
+ "tool_args": {
43
+ "action": "ax_action",
44
+ "target": {
45
+ "role": "AXButton",
46
+ "title": "OK"
47
+ },
48
+ "operation": "press"
49
+ }
50
+}
51
+```
52
+
53
+Supported operations are:
54
+
55
+- `press`: activate a button, menu item, checkbox, or similar control
56
+- `focus`: focus a text field or focusable element
57
+- `set_value`: set a value; pass `value` or `text`
58
+
59
+Targeting options:
60
+
61
+- Prefer a semantic `target` when the control has a stable title, description, value, identifier, role, or subrole.
62
+- Use a `path` returned by the latest `ax_snapshot` only while the UI is unchanged.
63
+- If an action reports ambiguity, take a fresh snapshot and narrow the target with role plus title/description/identifier.
64
+- If an action reports a missing target, take a fresh snapshot before trying coordinates.
65
+
66
+AX actions are attempts, not proof. They attach a fresh screenshot after state-changing actions; inspect that image before saying the requested outcome happened.
67
+
68
+## macOS Window Actions
69
+
70
+For active-app window tasks, macOS shortcuts are usually:
71
+
72
+- `Command+H`: hide the active app
73
+- `Command+M`: minimize the active window
74
+
75
+Treat these as attempts. After any hide or minimize shortcut, inspect the fresh screenshot. If the target window is still visible, say the attempt failed or switch strategy; do not type follow-up text into the focused app unless the screenshot clearly shows the intended target.
76
+
77
+When a visible close/minimize/full-screen button or menu item is accessible in the AX tree, prefer `ax_snapshot` plus `ax_action` over a small coordinate click.
78
+
79
+## Permissions
80
+
81
+macOS Screen Recording affects screenshots. Accessibility/Input Monitoring affect structural targeting and input. If `computer_use_remote` returns `COMPUTER_USE_REARM_REQUIRED`, `COMPUTER_USE_APPROVAL_REQUIRED`, or `status=rearm required`, stop immediately and ask the user to run `/computer-use on` in the A0 CLI and approve the macOS prompt if shown.
82
+
83
+Do not bypass a permission failure with server screenshots, Docker commands, `linux-desktop`, or browser fallbacks.
plugins/_a0_connector/skills/host-computer-use/SKILL.md
+14
-7
@@ -57,8 +57,9 @@ Use:
57
58
Arguments:
59
60
-- `action`: `start_session`, `status`, `capture`, `move`, `click`, `scroll`, `key`, `type`, `stop_session`
60
+- `action`: `start_session`, `status`, `capture`, `ax_snapshot`, `ax_action`, `move`, `click`, `scroll`, `key`, `type`, `stop_session`
61
- `session_id`: optional after `start_session`
62
+- `ax_snapshot`, `ax_action`: backend-gated structural accessibility actions; use only when backend metadata advertises matching support, and load the backend-specific skill first
63
- `move`: `x`, `y` normalized to `[0,1]`
64
- `click`: optional `x`, `y`, optional `button` (`left`, `right`, `middle`), optional `count`
65
- `scroll`: `dx`, `dy`
@@ -72,16 +73,22 @@ If any tool result contains `COMPUTER_USE_REARM_REQUIRED` or `status=rearm requi
73
## Core Loop
74
75
1. Call `start_session` first.
75
-2. Decide from the latest screenshot, not from memory.
76
-3. Interactive actions (`move`, `click`, `scroll`, `key`, `type`) already attach a fresh screenshot after they run; inspect it before claiming the requested outcome succeeded.
77
-4. Use `status` for state without starting a session.
78
-5. Use `capture` only when you need another screenshot without taking an action.
76
+2. Read the returned `backend_id`, `backend_family`, and `features`; load a backend-specific Computer Use skill when the task needs backend-only affordances.
77
+3. Decide final success from the latest screenshot, not from memory.
78
+4. Interactive actions (`ax_action`, `move`, `click`, `scroll`, `key`, `type`) already attach a fresh screenshot after they run; inspect it before claiming the requested outcome succeeded.
79
+5. Use `status` for state without starting a session.
80
+6. Use `capture` only when you need another screenshot without taking an action.
81
+
82
+## Backend Skills
83
+
84
+- If the backend is macOS or features include `accessibility-tree-snapshot` / `accessibility-structural-targeting`, load `host-computer-use-macos` before using `ax_snapshot` or `ax_action`.
85
+- Do not use backend-specific actions just because their argument names exist in the generic contract. Treat them as unavailable unless the connected CLI advertises the matching feature.
86
87
## Operating Rules
88
89
- Only the latest screenshot or a definitive tool result counts as evidence.
90
- If a tool result says a screenshot was attached but you cannot actually see the image, stop and report that visual verification is unavailable. Do not continue with another action from an assumed host state.
84
-- The current API uses normalized global screen coordinates; do not assume window ids, element indexes, background-safe input, or semantic click targets unless the runtime explicitly advertises them.
91
+- Outside advertised AX support, use normalized global screen coordinates; do not assume window ids, element indexes, background-safe input, or semantic click targets unless the runtime explicitly advertises them.
92
- Prefer accessibility and semantic UI paths first: shortcuts, command palettes, menu accelerators, address/search bars, focus traversal, and other keyboard-accessible controls.
93
- Prefer `key` and `type` over pointer actions whenever a reliable keyboard path exists.
94
- When a menu or popup is open, treat it as the active UI and prefer keyboard navigation over clicking small transient rows by coordinate.
@@ -101,7 +108,7 @@ If any tool result contains `COMPUTER_USE_REARM_REQUIRED` or `status=rearm requi
108
109
- Try keyboard scrolling first: `page_down`, `page_up`, `space`, `shift+space`, arrows, `home`, or `end`.
110
- Use `scroll` when the desired pane is already active or keyboard scrolling cannot target it.
104
-- Treat `move` and `click` as last-resort actions for controls that cannot be reached through keyboard, accessibility, browser, or app-native tooling.
111
+- Treat `move` and `click` as last-resort actions for controls that cannot be reached through backend-specific structural targeting, keyboard, browser, or app-native tooling.
112
- Before clicking, make sure the latest screenshot makes the target unambiguous. Use one deliberate click, then reassess from the fresh screenshot.
113
114
## Control Signals
plugins/_a0_connector/tools/computer_use_remote.py
+107
-11
@@ -34,6 +34,7 @@ REARM_REQUIRED_DEFAULT_MESSAGE = (
34
)
35
_AUTO_CAPTURE_ACTIONS = {
36
"start_session",
37
+ "ax_action",
38
"move",
39
"click",
40
"scroll",
@@ -46,6 +47,7 @@ _SETTLE_DELAY_CLICK = 0.35
47
_SETTLE_DELAY_SCROLL = 0.35
48
_SETTLE_DELAY_KEY = 0.2
49
_SETTLE_DELAY_TYPE = 0.25
50
+_SETTLE_DELAY_AX_ACTION = 0.25
51
_SETTLE_DELAY_GLOBAL_FOCUS = 0.45
52
_SETTLE_DELAY_PLAIN_ENTER = 0.3
53
_SETTLE_DELAY_SUBMIT = 0.45
@@ -54,6 +56,8 @@ _SUPPORTED_ACTIONS = {
56
"start_session",
57
"status",
58
"capture",
59
+ "ax_snapshot",
60
+ "ax_action",
61
"move",
62
"click",
63
"scroll",
@@ -72,7 +76,8 @@ class ComputerUseRemote(Tool):
76
return Response(
77
message=(
78
"action is required and must be one of: "
75
- "start_session, status, capture, move, click, scroll, key, type, stop_session"
79
+ "start_session, status, capture, ax_snapshot, ax_action, "
80
+ "move, click, scroll, key, type, stop_session"
81
),
82
break_loop=False,
83
)
@@ -255,6 +260,8 @@ class ComputerUseRemote(Tool):
260
return _SETTLE_DELAY_CLICK
261
if action == "scroll":
262
return _SETTLE_DELAY_SCROLL
263
+ if action == "ax_action":
264
+ return _SETTLE_DELAY_AX_ACTION
265
if action == "type" and self._coerce_bool(self.args.get("submit")):
266
return _SETTLE_DELAY_SUBMIT
267
if action == "type":
@@ -310,6 +317,24 @@ class ComputerUseRemote(Tool):
317
payload["text"] = self.args.get("text", "")
318
if self._coerce_bool(self.args.get("submit")):
319
payload["submit"] = True
320
+ elif action == "ax_snapshot":
321
+ if "max_depth" in self.args:
322
+ payload["max_depth"] = self._coerce_int(self.args.get("max_depth"), name="max_depth")
323
+ if "max_nodes" in self.args:
324
+ payload["max_nodes"] = self._coerce_int(self.args.get("max_nodes"), name="max_nodes")
325
+ elif action == "ax_action":
326
+ target = self.args.get("target")
327
+ if isinstance(target, dict):
328
+ payload["target"] = dict(target)
329
+ if "path" in self.args:
330
+ payload["path"] = self.args.get("path")
331
+ operation = self.args.get("operation", self.args.get("ax_action", self.args.get("name")))
332
+ if operation is not None:
333
+ payload["operation"] = operation
334
+ if "value" in self.args:
335
+ payload["value"] = self.args.get("value")
336
+ if "text" in self.args:
337
+ payload["text"] = self.args.get("text", "")
338
339
return payload
340
@@ -329,13 +354,27 @@ class ComputerUseRemote(Tool):
354
if action == "capture":
355
summary = self._record_capture(data)
356
return f"Current screen attached: {summary} {CAPTURE_VERIFICATION_NOTE}"
357
+ if action == "ax_snapshot":
358
+ return self._format_ax_snapshot(data)
359
+ if action == "ax_action":
360
+ target = data.get("target") if isinstance(data.get("target"), dict) else {}
361
+ operation = str(data.get("operation") or "?")
362
+ path = target.get("path", "?")
363
+ return f"Performed AX {operation} on {self._ax_target_label(target)} path={path}."
364
if action == "status":
365
return self._format_status(data)
366
if action == "start_session":
335
- return (
367
+ message = (
368
f"Computer-use session started: session_id={data.get('session_id', '?')} "
369
f"size={data.get('width', '?')}x{data.get('height', '?')}"
370
)
371
+ backend_details = self._format_backend_details(data)
372
+ if backend_details:
373
+ message = f"{message}, {backend_details}"
374
+ skill_hint = self._backend_skill_hint(data)
375
+ if skill_hint:
376
+ return f"{message}.{skill_hint}"
377
+ return message
378
if action == "stop_session":
379
return "Computer-use session stopped."
380
if action == "move":
@@ -373,14 +412,54 @@ class ComputerUseRemote(Tool):
412
return f"{code}: {error}"
413
return error
414
415
+ def _format_backend_details(self, data: dict[str, Any]) -> str:
416
+ backend_id = str(data.get("backend_id", "") or "").strip()
417
+ backend_family = str(data.get("backend_family", "") or "").strip()
418
+ features = self._backend_features(data)
419
+ parts: list[str] = []
420
+ if backend_id:
421
+ backend_text = backend_id
422
+ if backend_family:
423
+ backend_text = f"{backend_text}/{backend_family}"
424
+ parts.append(f"backend={backend_text}")
425
+ if features:
426
+ parts.append(f"features={', '.join(features)}")
427
+ return ", ".join(parts)
428
+
429
+ def _backend_features(self, data: dict[str, Any]) -> list[str]:
430
+ raw_features = data.get("features") or []
431
+ if not isinstance(raw_features, (list, tuple, set)):
432
+ return []
433
+ features: list[str] = []
434
+ for feature in raw_features:
435
+ text = str(feature or "").strip()
436
+ if text:
437
+ features.append(text)
438
+ return features
439
+
440
+ def _backend_skill_hint(self, data: dict[str, Any]) -> str:
441
+ backend_id = str(data.get("backend_id", "") or "").strip().lower()
442
+ backend_family = str(data.get("backend_family", "") or "").strip().lower()
443
+ features = {feature.lower() for feature in self._backend_features(data)}
444
+ has_macos_ax = bool(
445
+ features
446
+ & {
447
+ "accessibility-tree-snapshot",
448
+ "accessibility-structural-targeting",
449
+ }
450
+ )
451
+ if backend_id == "macos" or backend_family == "macos" or has_macos_ax:
452
+ return (
453
+ " Load skill `host-computer-use-macos` before using macOS AX "
454
+ "structural actions."
455
+ )
456
+ return ""
457
+
458
def _format_status(self, data: dict[str, Any]) -> str:
459
status = str(data.get("status", "unknown") or "unknown")
460
trust_mode = str(data.get("trust_mode", "") or "")
379
- backend_id = str(data.get("backend_id", "") or "").strip()
380
- backend_family = str(data.get("backend_family", "") or "").strip()
461
active_contexts = data.get("active_contexts") or []
462
active_text = ", ".join(str(item) for item in active_contexts) if active_contexts else "none"
383
- backend_text = ""
463
rearm_guidance = ""
464
if status == "rearm required":
465
detail = str(data.get("last_error") or "").strip()
@@ -394,20 +473,37 @@ class ComputerUseRemote(Tool):
473
"is not armed. "
474
"Stop using computer_use_remote until the user re-arms it."
475
)
397
- if backend_id:
398
- backend_text = backend_id
399
- if backend_family:
400
- backend_text = f"{backend_text}/{backend_family}"
401
- if backend_text:
476
+ backend_details = self._format_backend_details(data)
477
+ if backend_details:
478
return (
479
f"Computer use status={status}, trust_mode={trust_mode or 'unknown'}, "
404
- f"backend={backend_text}, active_contexts={active_text}.{rearm_guidance}"
480
+ f"{backend_details}, active_contexts={active_text}."
481
+ f"{self._backend_skill_hint(data)}{rearm_guidance}"
482
)
483
return (
484
f"Computer use status={status}, trust_mode={trust_mode or 'unknown'}, "
485
f"active_contexts={active_text}.{rearm_guidance}"
486
)
487
488
+ def _format_ax_snapshot(self, data: dict[str, Any]) -> str:
489
+ app = data.get("app") if isinstance(data.get("app"), dict) else {}
490
+ tree = data.get("tree") if isinstance(data.get("tree"), dict) else {}
491
+ app_name = str(app.get("name") or app.get("bundle_id") or "frontmost app")
492
+ node_count = data.get("node_count", "?")
493
+ truncated = " truncated" if data.get("truncated") else ""
494
+ root_label = self._ax_target_label(tree)
495
+ return (
496
+ f"AX snapshot for {app_name}: {node_count} node(s){truncated}. "
497
+ f"Root {root_label}. Use path or semantic target fields with ax_action."
498
+ )
499
+
500
+ def _ax_target_label(self, target: dict[str, Any]) -> str:
501
+ role = str(target.get("role") or "element")
502
+ title = str(target.get("title") or target.get("description") or target.get("identifier") or "").strip()
503
+ if title:
504
+ return f"{role} {title!r}"
505
+ return role
506
+
507
def _record_capture(self, data: dict[str, Any]) -> str:
508
display_ref, resolved_capture_id = self._resolve_capture_ref(data)
509
width = data.get("width", "?")
tests/test_a0_connector_prompt_gating.py
+22
@@ -345,6 +345,14 @@ def test_remote_affordance_skills_parse():
345
/ "host-computer-use"
346
/ "SKILL.md"
347
)
348
+ macos_computer_skill = _parse_skill_frontmatter(
349
+ PROJECT_ROOT
350
+ / "plugins"
351
+ / "_a0_connector"
352
+ / "skills"
353
+ / "host-computer-use-macos"
354
+ / "SKILL.md"
355
+ )
356
357
assert not legacy_connector_skill.exists()
358
assert text_editor_skill["name"] == "host-file-editing"
@@ -358,6 +366,8 @@ def test_remote_affordance_skills_parse():
366
assert "Use instead of linux-desktop" in computer_skill["description"]
367
assert "host computer" in computer_skill["triggers"]
368
assert "Ubuntu Wayland desktop" in computer_skill["triggers"]
369
+ assert macos_computer_skill["name"] == "host-computer-use-macos"
370
+ assert "macOS guidance" in macos_computer_skill["description"]
371
372
373
def test_remote_tool_stubs_are_self_contained_and_reference_per_tool_skills():
@@ -372,6 +382,14 @@ def test_remote_tool_stubs_are_self_contained_and_reference_per_tool_skills():
382
/ "host-computer-use"
383
/ "SKILL.md"
384
).read_text(encoding="utf-8")
385
+ macos_computer_skill = (
386
+ PROJECT_ROOT
387
+ / "plugins"
388
+ / "_a0_connector"
389
+ / "skills"
390
+ / "host-computer-use-macos"
391
+ / "SKILL.md"
392
+ ).read_text(encoding="utf-8")
393
394
assert "optionally load skill `host-file-editing`" in text_stub
395
assert "optionally load skill `host-code-execution`" in exec_stub
@@ -379,8 +397,12 @@ def test_remote_tool_stubs_are_self_contained_and_reference_per_tool_skills():
397
assert '"tool_name": "code_execution_remote"' in exec_stub
398
assert '"tool_name": "computer_use_remote"' in computer_stub
399
assert "load and follow skill `host-computer-use`" in computer_stub
400
+ assert "host-computer-use-macos" in computer_stub
401
assert "Do not substitute the `linux-desktop` skill" in computer_stub
402
assert '"tool_name": "computer_use_remote"' in computer_skill
403
+ assert '"tool_name": "computer_use_remote"' in macos_computer_skill
404
+ assert "ax_snapshot" in macos_computer_skill
405
+ assert "ax_snapshot`/`ax_action` are structural Accessibility targeting" not in computer_skill
406
assert "Availability, backend support, and trust mode are checked when the tool runs" in computer_stub
407
assert "not `code_execution_tool`" in exec_stub
408
assert "not to" in exec_stub
tests/test_skills_runtime.py
+1
@@ -291,6 +291,7 @@ def test_renamed_skills_use_standard_frontmatter_only():
291
PROJECT_ROOT / "skills" / "scheduled-tasks" / "SKILL.md",
292
PROJECT_ROOT / "plugins" / "_a0_connector" / "skills" / "host-code-execution" / "SKILL.md",
293
PROJECT_ROOT / "plugins" / "_a0_connector" / "skills" / "host-computer-use" / "SKILL.md",
294
+ PROJECT_ROOT / "plugins" / "_a0_connector" / "skills" / "host-computer-use-macos" / "SKILL.md",
295
PROJECT_ROOT / "plugins" / "_a0_connector" / "skills" / "host-file-editing" / "SKILL.md",
296
PROJECT_ROOT / "plugins" / "_a0_connector" / "skills" / "setup-a0-cli" / "SKILL.md",
297
PROJECT_ROOT / "plugins" / "_browser" / "skills" / "browser-automation" / "SKILL.md",
tests/test_tool_action_contracts.py
+79
@@ -86,6 +86,50 @@ def _load_skills_tool(monkeypatch, skill_root: Path):
86
return importlib.import_module("tools.skills_tool")
87
88
89
+def _load_computer_use_remote_tool(monkeypatch):
90
+ _install_tool_stub(monkeypatch)
91
+
92
+ history_stub = types.ModuleType("helpers.history")
93
+
94
+ class _RawMessage(dict):
95
+ def __init__(self, raw_content, preview):
96
+ super().__init__(raw_content=raw_content, preview=preview)
97
+
98
+ history_stub.RawMessage = _RawMessage
99
+ monkeypatch.setitem(sys.modules, "helpers.history", history_stub)
100
+
101
+ print_style_stub = types.ModuleType("helpers.print_style")
102
+ print_style_stub.PrintStyle = lambda *args, **kwargs: types.SimpleNamespace(
103
+ print=lambda *a, **k: None
104
+ )
105
+ monkeypatch.setitem(sys.modules, "helpers.print_style", print_style_stub)
106
+
107
+ ws_stub = types.ModuleType("helpers.ws")
108
+ ws_stub.NAMESPACE = "/test"
109
+ monkeypatch.setitem(sys.modules, "helpers.ws", ws_stub)
110
+
111
+ ws_manager_stub = types.ModuleType("helpers.ws_manager")
112
+ ws_manager_stub.ConnectionNotFoundError = RuntimeError
113
+ ws_manager_stub.get_shared_ws_manager = lambda: types.SimpleNamespace(
114
+ emit_to=lambda *a, **k: None
115
+ )
116
+ monkeypatch.setitem(sys.modules, "helpers.ws_manager", ws_manager_stub)
117
+
118
+ ws_runtime_stub = types.ModuleType("plugins._a0_connector.helpers.ws_runtime")
119
+ ws_runtime_stub.clear_pending_computer_use_op = lambda *args, **kwargs: None
120
+ ws_runtime_stub.computer_use_metadata_for_sid = lambda *args, **kwargs: {}
121
+ ws_runtime_stub.select_computer_use_target_sid = lambda *args, **kwargs: "sid"
122
+ ws_runtime_stub.store_pending_computer_use_op = lambda *args, **kwargs: None
123
+ monkeypatch.setitem(
124
+ sys.modules,
125
+ "plugins._a0_connector.helpers.ws_runtime",
126
+ ws_runtime_stub,
127
+ )
128
+
129
+ sys.modules.pop("plugins._a0_connector.tools.computer_use_remote", None)
130
+ return importlib.import_module("plugins._a0_connector.tools.computer_use_remote")
131
+
132
+
133
def test_skills_tool_accepts_action_alias_for_search(monkeypatch, tmp_path: Path):
134
module = _load_skills_tool(monkeypatch, tmp_path)
135
tool = module.SkillsTool(
@@ -559,6 +603,10 @@ def test_computer_use_remote_is_runtime_checked_standard_tool():
603
project_root
604
/ "plugins/_a0_connector/skills/host-computer-use/SKILL.md"
605
).read_text(encoding="utf-8")
606
+ macos_skill_text = (
607
+ project_root
608
+ / "plugins/_a0_connector/skills/host-computer-use-macos/SKILL.md"
609
+ ).read_text(encoding="utf-8")
610
611
assert standard_prompt_path.exists()
612
assert not (
@@ -569,5 +617,36 @@ def test_computer_use_remote_is_runtime_checked_standard_tool():
617
assert "not scoped to a single chat context" in standard_prompt_text
618
assert "checked when the tool runs" in standard_prompt_text
619
assert "visual verification is unavailable" in standard_prompt_text
620
+ assert "host-computer-use-macos" in standard_prompt_text
621
assert '"tool_name": "computer_use_remote"' in skill_text
622
+ assert '"tool_name": "computer_use_remote"' in macos_skill_text
623
+ assert "Backend-specific macOS guidance" in macos_skill_text
624
assert "Beta desktop control" in skill_text
625
+
626
+
627
+def test_computer_use_remote_start_session_reports_backend_features_and_macos_skill(monkeypatch):
628
+ module = _load_computer_use_remote_tool(monkeypatch)
629
+ tool = object.__new__(module.ComputerUseRemote)
630
+
631
+ message = tool._extract_result(
632
+ "start_session",
633
+ {
634
+ "ok": True,
635
+ "result": {
636
+ "session_id": "s1",
637
+ "width": 1920,
638
+ "height": 1080,
639
+ "backend_id": "macos",
640
+ "backend_family": "macos",
641
+ "features": [
642
+ "accessibility-tree-snapshot",
643
+ "accessibility-structural-targeting",
644
+ ],
645
+ },
646
+ },
647
+ )
648
+
649
+ assert "session_id=s1" in message
650
+ assert "backend=macos/macos" in message
651
+ assert "features=accessibility-tree-snapshot, accessibility-structural-targeting" in message
652
+ assert "host-computer-use-macos" in message