@samitouri / QOSami_Agents_Zero / commits / 62ac20e7

feat(desktop): add Linux Desktop skill controls

Add a linux-desktop skill that teaches Agent Zero how to operate the persistent XFCE/Xpra desktop through desktopctl.sh, including app launch, focus, click, typing, and stable folder entry points for Workdir, Projects, Skills, Agents, and Downloads. Add a Calc cell-edit helper that opens a workbook through the visible LibreOffice Calc desktop session, updates a requested sheet cell, saves, and verifies the XLSX on disk. Expand the Office canvas setup tests to cover Desktop branding, Xpra package requirements, resize behavior, mobile canvas gating, and the new skill helpers.

Alessandro committed May 2, 2026 at 12:21 UTC 62ac20e7b248179825e05664c1df97ebc6214c54
4 files changed +880 -7
plugins/_office/skills/linux-desktop/SKILL.md new
+110
@@ -0,0 +1,110 @@
1 +---
2 +name: linux-desktop
3 +description: Use when the user asks Agent Zero to operate the built-in Linux Desktop, XFCE apps, LibreOffice GUI apps, file manager, terminal, or visual desktop workflows.
4 +version: "0.2.0"
5 +author: "Agent Zero Core Team"
6 +tags: ["linux", "desktop", "xfce", "libreoffice", "gui", "files", "terminal"]
7 +triggers:
8 + - "Linux desktop"
9 + - "Desktop GUI"
10 + - "XFCE"
11 + - "LibreOffice GUI"
12 + - "file manager"
13 + - "terminal app"
14 + - "use the OS"
15 +allowed_tools:
16 + - document_artifact
17 + - code_execution_tool
18 +---
19 +
20 +# Linux Desktop Interface
21 +
22 +Use the Desktop as a full Linux GUI when the user needs a visual workflow, an installed desktop app, or manual layout polish that is awkward through structured file edits alone. Use structured tools first for deterministic content changes, then use the Desktop for inspection, GUI-only actions, and final visual confirmation.
23 +
24 +## Operating Model
25 +
26 +1. Prefer `document_artifact` for creating, reading, and editing Markdown, DOCX, XLSX, and PPTX files.
27 +2. Open the Desktop only when the user asks for the Desktop, a GUI app, or visual confirmation.
28 +3. Launch common apps from the Desktop icons, the header buttons, or `scripts/desktopctl.sh`.
29 +4. Use the external Agent Zero Browser for web browsing. Do not launch an operating-system browser in this version.
30 +5. Verify GUI work by observing the desktop state, checking window titles, and saving the file before reporting success.
31 +
32 +## Control Flow
33 +
34 +Use the helper script when the Desktop is already open and you need reliable app launches, clicks, keystrokes, or window checks from the agent shell:
35 +
36 +```bash
37 +plugins/_office/skills/linux-desktop/scripts/desktopctl.sh check
38 +plugins/_office/skills/linux-desktop/scripts/desktopctl.sh launch calc
39 +plugins/_office/skills/linux-desktop/scripts/desktopctl.sh windows LibreOffice
40 +plugins/_office/skills/linux-desktop/scripts/desktopctl.sh focus LibreOffice
41 +plugins/_office/skills/linux-desktop/scripts/desktopctl.sh key ctrl+s
42 +```
43 +
44 +The script targets the persistent `agent-zero-desktop` X display, sets `DISPLAY`, `XAUTHORITY`, and `HOME` to the XFCE profile, then uses `xdotool` for input. If `check` fails, ask the user to open the Desktop surface or open it from the UI before attempting X11 automation.
45 +
46 +For direct app launches without coordinates:
47 +
48 +```bash
49 +plugins/_office/skills/linux-desktop/scripts/desktopctl.sh launch writer
50 +plugins/_office/skills/linux-desktop/scripts/desktopctl.sh launch calc
51 +plugins/_office/skills/linux-desktop/scripts/desktopctl.sh launch impress
52 +plugins/_office/skills/linux-desktop/scripts/desktopctl.sh launch terminal
53 +plugins/_office/skills/linux-desktop/scripts/desktopctl.sh launch settings
54 +plugins/_office/skills/linux-desktop/scripts/desktopctl.sh open-path /a0/usr/workdir
55 +```
56 +
57 +For live spreadsheet coworking, use the Calc helper instead of hand-written UNO snippets:
58 +
59 +```bash
60 +plugins/_office/skills/linux-desktop/scripts/desktopctl.sh calc-set-cell /a0/usr/workdir/example.xlsx Sheet1 B2 "Cowork verified live"
61 +```
62 +
63 +This opens the workbook in the visible Desktop Calc session if needed, changes the cell through LibreOffice, saves the workbook, and verifies the `.xlsx` on disk. Because the edit happens through the running LibreOffice session, the user can see the sheet update without refreshing the Desktop surface.
64 +
65 +For coordinate actions after observing the Desktop:
66 +
67 +```bash
68 +plugins/_office/skills/linux-desktop/scripts/desktopctl.sh click 120 180
69 +plugins/_office/skills/linux-desktop/scripts/desktopctl.sh dblclick 120 180
70 +plugins/_office/skills/linux-desktop/scripts/desktopctl.sh type "Text to enter"
71 +plugins/_office/skills/linux-desktop/scripts/desktopctl.sh location
72 +```
73 +
74 +When browser automation is available, the higher-level QA flow is:
75 +
76 +1. Open `http://127.0.0.1:32080`.
77 +2. Open the Desktop canvas from the UI or with `Alpine.store("rightCanvas").open("office")`.
78 +3. Use browser mouse events into the Xpra iframe for real user-path testing.
79 +4. Cross-check with `desktopctl.sh location` and `desktopctl.sh windows PATTERN`.
80 +5. Capture the browser screenshot as visual evidence.
81 +
82 +## Desktop Locations
83 +
84 +The Desktop exposes stable folders for common user work:
85 +
86 +- `Workdir` -> `/a0/usr/workdir`
87 +- `Projects` -> `/a0/usr/projects`
88 +- `Skills` -> `/a0/usr/skills`
89 +- `Agents` -> `/a0/usr/agents`
90 +- `Downloads` -> `/a0/usr/downloads`
91 +
92 +Use these folders when the user asks to inspect or manipulate project files, skills, agent profiles, or downloaded artifacts from the GUI.
93 +
94 +## App Map
95 +
96 +- `LibreOffice Writer`: word processing and DOCX layout.
97 +- `LibreOffice Calc`: spreadsheets, formulas, tables, charts.
98 +- `LibreOffice Impress`: presentations and slide polish.
99 +- `Workdir`: graphical file management with Thunar at `/a0/usr/workdir`.
100 +- `Terminal`: shell work inside the Agent Zero runtime.
101 +- `Settings`: XFCE system settings.
102 +
103 +## Practical Rules
104 +
105 +- Keep installs inside normal plugin hook or image install flows; do not install packages ad hoc just to complete one desktop action.
106 +- Do not treat closing a document tab as closing the whole Desktop. The Desktop is persistent while Agent Zero is running.
107 +- Save before sync or final verification when the GUI app has edited a file.
108 +- If a GUI action is flaky, switch to structured file editing for content and return to the Desktop only for visual confirmation.
109 +- For live Calc edits that the user should see immediately, prefer `desktopctl.sh calc-set-cell FILE SHEET CELL VALUE`.
110 +- For enterprise workflows, leave printing available.
plugins/_office/skills/linux-desktop/scripts/calc_set_cell.py new
+315
@@ -0,0 +1,315 @@
1 +#!/usr/bin/env python3
2 +"""Set a Calc cell through the visible LibreOffice desktop session."""
3 +
4 +from __future__ import annotations
5 +
6 +import argparse
7 +import os
8 +import shutil
9 +import subprocess
10 +import sys
11 +import time
12 +import zipfile
13 +from pathlib import Path
14 +import xml.etree.ElementTree as ET
15 +
16 +for candidate in ("/usr/lib/python3/dist-packages", "/usr/lib/libreoffice/program"):
17 + if candidate not in sys.path:
18 + sys.path.insert(0, candidate)
19 +
20 +try:
21 + import uno
22 +except Exception as exc: # pragma: no cover - exercised in the container runtime.
23 + raise SystemExit(f"LibreOffice UNO is not available: {exc}") from exc
24 +
25 +
26 +UNO_URL = "uno:socket,host=localhost,port=2002;urp;StarOffice.ComponentContext"
27 +OOXML_MAIN_NS = "http://schemas.openxmlformats.org/spreadsheetml/2006/main"
28 +OOXML_REL_NS = "http://schemas.openxmlformats.org/officeDocument/2006/relationships"
29 +REL_PACKAGE_NS = "http://schemas.openxmlformats.org/package/2006/relationships"
30 +NS = {"m": OOXML_MAIN_NS, "r": OOXML_REL_NS, "rel": REL_PACKAGE_NS}
31 +
32 +
33 +def parse_args() -> argparse.Namespace:
34 + parser = argparse.ArgumentParser(
35 + description="Open an XLSX in visible LibreOffice Calc, edit one cell, save, and verify."
36 + )
37 + parser.add_argument("file")
38 + parser.add_argument("sheet")
39 + parser.add_argument("cell")
40 + parser.add_argument("value", nargs=argparse.REMAINDER)
41 + args = parser.parse_args()
42 + if not args.value:
43 + parser.error("value is required")
44 + args.value = " ".join(args.value)
45 + return args
46 +
47 +
48 +def run_quiet(command: list[str], *, timeout: float | None = None) -> subprocess.CompletedProcess:
49 + return subprocess.run(
50 + command,
51 + stdout=subprocess.DEVNULL,
52 + stderr=subprocess.DEVNULL,
53 + check=False,
54 + timeout=timeout,
55 + )
56 +
57 +
58 +def dismiss_document_in_use_dialogs() -> None:
59 + xdotool = shutil.which("xdotool")
60 + if not xdotool:
61 + return
62 + found = subprocess.run(
63 + [xdotool, "search", "--onlyvisible", "--name", "Document in Use"],
64 + stdout=subprocess.PIPE,
65 + stderr=subprocess.DEVNULL,
66 + text=True,
67 + check=False,
68 + )
69 + for window_id in found.stdout.splitlines():
70 + if window_id.strip():
71 + run_quiet([xdotool, "windowclose", window_id.strip()], timeout=2)
72 +
73 +
74 +def connect_once():
75 + local_ctx = uno.getComponentContext()
76 + resolver = local_ctx.ServiceManager.createInstanceWithContext(
77 + "com.sun.star.bridge.UnoUrlResolver",
78 + local_ctx,
79 + )
80 + return resolver.resolve(os.environ.get("A0_DESKTOP_UNO_URL", UNO_URL))
81 +
82 +
83 +def desktop_from_context(ctx):
84 + return ctx.ServiceManager.createInstanceWithContext("com.sun.star.frame.Desktop", ctx)
85 +
86 +
87 +def component_url(component) -> str:
88 + try:
89 + return getattr(component, "URL", "") or ""
90 + except Exception:
91 + return ""
92 +
93 +
94 +def component_title(component) -> str:
95 + try:
96 + return getattr(component, "Title", "") or ""
97 + except Exception:
98 + return ""
99 +
100 +
101 +def component_has_sheets(component) -> bool:
102 + try:
103 + component.getSheets()
104 + return True
105 + except Exception:
106 + return False
107 +
108 +
109 +def url_to_path(url: str) -> str:
110 + if not url:
111 + return ""
112 + try:
113 + return os.path.realpath(uno.fileUrlToSystemPath(url))
114 + except Exception:
115 + return ""
116 +
117 +
118 +def find_open_document(desktop, path: Path):
119 + target_path = os.path.realpath(str(path))
120 + target_name = path.name
121 + try:
122 + enum = desktop.getComponents().createEnumeration()
123 + except Exception:
124 + return None
125 + while enum.hasMoreElements():
126 + component = enum.nextElement()
127 + if not component_has_sheets(component):
128 + continue
129 + if url_to_path(component_url(component)) == target_path:
130 + return component
131 + if target_name and target_name in component_title(component):
132 + return component
133 + return None
134 +
135 +
136 +def libreoffice_command(path: Path) -> list[str]:
137 + soffice = os.environ.get("SOFFICE") or shutil.which("soffice") or "soffice"
138 + home = os.environ.get("HOME", "")
139 + user_installation = f"-env:UserInstallation=file://{home}" if home else ""
140 + command = [
141 + soffice,
142 + "--norestore",
143 + "--nofirststartwizard",
144 + "--nolockcheck",
145 + "--accept=socket,host=localhost,port=2002;urp;StarOffice.ComponentContext",
146 + "--calc",
147 + str(path),
148 + ]
149 + if user_installation:
150 + command.insert(4, user_installation)
151 + return command
152 +
153 +
154 +def launch_visible_calc(path: Path) -> None:
155 + subprocess.Popen(
156 + libreoffice_command(path),
157 + stdout=subprocess.DEVNULL,
158 + stderr=subprocess.DEVNULL,
159 + start_new_session=True,
160 + )
161 +
162 +
163 +def wait_for_document(path: Path, timeout_seconds: float):
164 + started = False
165 + last_error: Exception | None = None
166 + deadline = time.time() + timeout_seconds
167 + while time.time() < deadline:
168 + dismiss_document_in_use_dialogs()
169 + try:
170 + ctx = connect_once()
171 + desktop = desktop_from_context(ctx)
172 + document = find_open_document(desktop, path)
173 + if document is not None:
174 + return document
175 + if not started:
176 + launch_visible_calc(path)
177 + started = True
178 + except Exception as exc:
179 + last_error = exc
180 + if not started:
181 + launch_visible_calc(path)
182 + started = True
183 + time.sleep(0.5)
184 + detail = f": {last_error!r}" if last_error else ""
185 + raise RuntimeError(f"Timed out waiting for LibreOffice Calc to open {path}{detail}")
186 +
187 +
188 +def focus_document_window(document) -> None:
189 + try:
190 + document.CurrentController.Frame.ContainerWindow.setFocus()
191 + except Exception:
192 + pass
193 +
194 +
195 +def sheet_path_for_name(zip_file: zipfile.ZipFile, sheet_name: str) -> str:
196 + workbook = ET.fromstring(zip_file.read("xl/workbook.xml"))
197 + rel_id = None
198 + sheets = workbook.find("m:sheets", NS)
199 + if sheets is None:
200 + raise RuntimeError("Workbook has no sheets collection")
201 + for sheet in sheets:
202 + if sheet.attrib.get("name") == sheet_name:
203 + rel_id = sheet.attrib.get(f"{{{OOXML_REL_NS}}}id")
204 + break
205 + if not rel_id:
206 + raise RuntimeError(f"Sheet not found on disk: {sheet_name}")
207 + rels = ET.fromstring(zip_file.read("xl/_rels/workbook.xml.rels"))
208 + target = None
209 + for rel in rels:
210 + if rel.attrib.get("Id") == rel_id:
211 + target = rel.attrib.get("Target")
212 + break
213 + if not target:
214 + raise RuntimeError(f"Sheet relationship not found on disk: {sheet_name}")
215 + target = target.lstrip("/")
216 + if target.startswith("xl/"):
217 + return target
218 + return f"xl/{target}"
219 +
220 +
221 +def shared_strings(zip_file: zipfile.ZipFile) -> list[str]:
222 + if "xl/sharedStrings.xml" not in zip_file.namelist():
223 + return []
224 + root = ET.fromstring(zip_file.read("xl/sharedStrings.xml"))
225 + values: list[str] = []
226 + for item in root.findall("m:si", NS):
227 + parts = [
228 + text_node.text or ""
229 + for text_node in item.iter(f"{{{OOXML_MAIN_NS}}}t")
230 + ]
231 + values.append("".join(parts))
232 + return values
233 +
234 +
235 +def read_xlsx_cell(path: Path, sheet_name: str, cell_name: str) -> str | None:
236 + with zipfile.ZipFile(path) as zip_file:
237 + sheet_path = sheet_path_for_name(zip_file, sheet_name)
238 + sheet = ET.fromstring(zip_file.read(sheet_path))
239 + strings = shared_strings(zip_file)
240 + for cell in sheet.iter(f"{{{OOXML_MAIN_NS}}}c"):
241 + if cell.attrib.get("r") != cell_name:
242 + continue
243 + cell_type = cell.attrib.get("t")
244 + if cell_type == "inlineStr":
245 + return "".join(
246 + text_node.text or ""
247 + for text_node in cell.iter(f"{{{OOXML_MAIN_NS}}}t")
248 + )
249 + value_node = cell.find("m:v", NS)
250 + raw_value = "" if value_node is None or value_node.text is None else value_node.text
251 + if cell_type == "s" and raw_value != "":
252 + return strings[int(raw_value)]
253 + return raw_value
254 + return None
255 +
256 +
257 +def wait_for_disk_value(path: Path, sheet: str, cell: str, expected: str) -> str | None:
258 + latest = None
259 + for _ in range(24):
260 + latest = read_xlsx_cell(path, sheet, cell)
261 + if latest == expected:
262 + return latest
263 + time.sleep(0.25)
264 + return latest
265 +
266 +
267 +def is_disposed_error(exc: BaseException) -> bool:
268 + return "DisposedException" in type(exc).__name__ or "DisposedException" in str(exc)
269 +
270 +
271 +def edit_visible_calc_cell(path: Path, sheet_name: str, cell_name: str, value: str) -> tuple[str, str, str | None]:
272 + last_error: Exception | None = None
273 + for _ in range(4):
274 + document = wait_for_document(
275 + path,
276 + timeout_seconds=float(os.environ.get("A0_CALC_OPEN_TIMEOUT", "30")),
277 + )
278 + try:
279 + focus_document_window(document)
280 + sheets = document.getSheets()
281 + if not sheets.hasByName(sheet_name):
282 + raise SystemExit(f"Sheet not found: {sheet_name}")
283 + sheet = sheets.getByName(sheet_name)
284 + target_cell = sheet.getCellRangeByName(cell_name)
285 + before = target_cell.getString()
286 + target_cell.setString(value)
287 + document.store()
288 + after = target_cell.getString()
289 + disk_value = wait_for_disk_value(path, sheet_name, cell_name, value)
290 + return before, after, disk_value
291 + except Exception as exc:
292 + if not is_disposed_error(exc):
293 + raise
294 + last_error = exc
295 + time.sleep(1)
296 + raise RuntimeError(f"LibreOffice UNO bridge stayed disposed while editing {path}: {last_error!r}")
297 +
298 +
299 +def main() -> int:
300 + args = parse_args()
301 + path = Path(args.file).expanduser().resolve()
302 + if not path.exists():
303 + raise SystemExit(f"Workbook not found: {path}")
304 +
305 + before, after, disk_value = edit_visible_calc_cell(path, args.sheet, args.cell, args.value)
306 + if after != args.value or disk_value != args.value:
307 + raise SystemExit(
308 + f"Calc edit verification failed: before={before!r} after={after!r} disk={disk_value!r}"
309 + )
310 + print(f"updated {path} {args.sheet}!{args.cell}: {before!r} -> {after!r}")
311 + return 0
312 +
313 +
314 +if __name__ == "__main__":
315 + raise SystemExit(main())
plugins/_office/skills/linux-desktop/scripts/desktopctl.sh new
+221
@@ -0,0 +1,221 @@
1 +#!/usr/bin/env bash
2 +set -euo pipefail
3 +
4 +SESSION="${A0_DESKTOP_SESSION:-agent-zero-desktop}"
5 +BASE_DIR="${A0_BASE_DIR:-/a0}"
6 +PROFILE_DIR="${A0_DESKTOP_PROFILE:-$BASE_DIR/tmp/_office/desktop/profiles/$SESSION}"
7 +MANIFEST="${A0_DESKTOP_MANIFEST:-$BASE_DIR/tmp/_office/desktop/sessions/$SESSION.json}"
8 +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
9 +
10 +display_from_manifest() {
11 + if [ ! -f "$MANIFEST" ] || ! command -v python3 >/dev/null 2>&1; then
12 + return 0
13 + fi
14 + python3 - "$MANIFEST" <<'PY'
15 +import json
16 +import sys
17 +
18 +try:
19 + with open(sys.argv[1], "r", encoding="utf-8") as handle:
20 + value = json.load(handle).get("display", "")
21 +except Exception:
22 + value = ""
23 +if value != "":
24 + print(value)
25 +PY
26 +}
27 +
28 +DISPLAY_VALUE="${A0_DESKTOP_DISPLAY:-$(display_from_manifest || true)}"
29 +DISPLAY_VALUE="${DISPLAY_VALUE:-120}"
30 +case "$DISPLAY_VALUE" in
31 + :*) export DISPLAY="$DISPLAY_VALUE" ;;
32 + *) export DISPLAY=":$DISPLAY_VALUE" ;;
33 +esac
34 +
35 +export XAUTHORITY="${A0_DESKTOP_XAUTHORITY:-$PROFILE_DIR/.Xauthority}"
36 +export HOME="${A0_DESKTOP_HOME:-$PROFILE_DIR}"
37 +export XDG_CONFIG_HOME="${XDG_CONFIG_HOME:-$HOME/.config}"
38 +export XDG_DATA_HOME="${XDG_DATA_HOME:-$HOME/.local/share}"
39 +export XDG_CACHE_HOME="${XDG_CACHE_HOME:-$HOME/.cache}"
40 +export XDG_CURRENT_DESKTOP="${XDG_CURRENT_DESKTOP:-XFCE}"
41 +
42 +command_name="${1:-help}"
43 +shift || true
44 +
45 +usage() {
46 + cat <<'EOF'
47 +Usage: desktopctl.sh <command> [args]
48 +
49 +Commands:
50 + env Print the X11 environment used for the Desktop.
51 + check Verify that xdotool can reach the Desktop display.
52 + location Print the current X pointer location.
53 + windows [PATTERN] List visible window names matching PATTERN.
54 + focus PATTERN Focus the first visible window matching PATTERN.
55 + key KEY... Send one or more xdotool key names.
56 + type TEXT Type text into the focused window.
57 + click X Y Move and click at X,Y in Desktop coordinates.
58 + dblclick X Y Move and double-click at X,Y in Desktop coordinates.
59 + launch APP Launch writer, calc, impress, terminal, settings, or workdir.
60 + open-path [PATH] Open PATH in Thunar, defaulting to /a0/usr/workdir.
61 + calc-set-cell FILE SHEET CELL VALUE
62 + Open FILE in visible Calc, set SHEET!CELL, save, and verify.
63 + save Send Ctrl+S to the focused app.
64 +EOF
65 +}
66 +
67 +require_xdotool() {
68 + if ! command -v xdotool >/dev/null 2>&1; then
69 + echo "xdotool is required for Desktop control." >&2
70 + exit 2
71 + fi
72 +}
73 +
74 +ensure_display() {
75 + require_xdotool
76 + if ! xdotool getmouselocation >/dev/null 2>&1; then
77 + echo "Desktop X display is not reachable. Open the Desktop surface first." >&2
78 + exit 2
79 + fi
80 +}
81 +
82 +run_detached() {
83 + ( "$@" >/tmp/a0-desktopctl.log 2>&1 & )
84 +}
85 +
86 +close_blocking_dialogs() {
87 + require_xdotool
88 + for title in "Document in Use"; do
89 + window_ids="$(xdotool search --onlyvisible --name "$title" 2>/dev/null || true)"
90 + printf '%s\n' "$window_ids" | while read -r window_id; do
91 + [ -n "$window_id" ] && xdotool windowclose "$window_id" >/dev/null 2>&1 || true
92 + done
93 + done
94 +}
95 +
96 +first_window() {
97 + pattern="$1"
98 + xdotool search --onlyvisible --name "$pattern" 2>/dev/null | head -n 1 || true
99 +}
100 +
101 +launch_app() {
102 + app="${1:-}"
103 + soffice="${SOFFICE:-$(command -v soffice || true)}"
104 + soffice="${soffice:-soffice}"
105 + case "$app" in
106 + writer)
107 + run_detached "$soffice" --norestore --nofirststartwizard --nolockcheck "-env:UserInstallation=file://$HOME" --writer
108 + ;;
109 + calc|spreadsheet)
110 + run_detached "$soffice" --norestore --nofirststartwizard --nolockcheck "-env:UserInstallation=file://$HOME" --calc
111 + ;;
112 + impress|presentation)
113 + run_detached "$soffice" --norestore --nofirststartwizard --nolockcheck "-env:UserInstallation=file://$HOME" --impress
114 + ;;
115 + terminal)
116 + run_detached xfce4-terminal --working-directory=/a0/usr/workdir
117 + ;;
118 + settings)
119 + run_detached xfce4-settings-manager
120 + ;;
121 + workdir|files|file-manager)
122 + run_detached thunar /a0/usr/workdir
123 + ;;
124 + *)
125 + echo "Unknown app: ${app:-<empty>}" >&2
126 + echo "Expected: writer, calc, impress, terminal, settings, or workdir." >&2
127 + exit 2
128 + ;;
129 + esac
130 +}
131 +
132 +case "$command_name" in
133 + help|-h|--help)
134 + usage
135 + ;;
136 + env)
137 + printf 'export DISPLAY=%q\n' "$DISPLAY"
138 + printf 'export XAUTHORITY=%q\n' "$XAUTHORITY"
139 + printf 'export HOME=%q\n' "$HOME"
140 + ;;
141 + check)
142 + ensure_display
143 + xdotool getmouselocation --shell
144 + ;;
145 + location)
146 + ensure_display
147 + xdotool getmouselocation --shell
148 + ;;
149 + windows)
150 + ensure_display
151 + pattern="${1:-.}"
152 + xdotool search --onlyvisible --name "$pattern" getwindowname %@ 2>/dev/null || true
153 + ;;
154 + focus)
155 + ensure_display
156 + pattern="${1:?focus requires a window name pattern}"
157 + window_id="$(first_window "$pattern")"
158 + if [ -z "$window_id" ]; then
159 + echo "No visible window matched: $pattern" >&2
160 + exit 1
161 + fi
162 + xdotool windowactivate --sync "$window_id"
163 + ;;
164 + key)
165 + ensure_display
166 + if [ "$#" -eq 0 ]; then
167 + echo "key requires at least one xdotool key name." >&2
168 + exit 2
169 + fi
170 + xdotool key --clearmodifiers "$@"
171 + ;;
172 + type)
173 + ensure_display
174 + text="$*"
175 + xdotool type --delay "${A0_DESKTOP_TYPE_DELAY_MS:-1}" -- "$text"
176 + ;;
177 + click)
178 + ensure_display
179 + x="${1:?click requires X}"
180 + y="${2:?click requires Y}"
181 + xdotool mousemove --sync "$x" "$y" click 1
182 + ;;
183 + dblclick)
184 + ensure_display
185 + x="${1:?dblclick requires X}"
186 + y="${2:?dblclick requires Y}"
187 + xdotool mousemove --sync "$x" "$y" click --repeat 2 --delay "${A0_DESKTOP_DBLCLICK_DELAY_MS:-150}" 1
188 + ;;
189 + launch)
190 + ensure_display
191 + launch_app "${1:-}"
192 + ;;
193 + open-path)
194 + ensure_display
195 + path="${1:-/a0/usr/workdir}"
196 + run_detached thunar "$path"
197 + ;;
198 + calc-set-cell)
199 + ensure_display
200 + file="${1:?calc-set-cell requires FILE}"
201 + sheet="${2:?calc-set-cell requires SHEET}"
202 + cell="${3:?calc-set-cell requires CELL}"
203 + shift 3
204 + if [ "$#" -eq 0 ]; then
205 + echo "calc-set-cell requires VALUE." >&2
206 + exit 2
207 + fi
208 + close_blocking_dialogs
209 + export PYTHONPATH="${PYTHONPATH:-/usr/lib/python3/dist-packages:/usr/lib/libreoffice/program:}"
210 + python3 "$SCRIPT_DIR/calc_set_cell.py" "$file" "$sheet" "$cell" "$@"
211 + ;;
212 + save)
213 + ensure_display
214 + xdotool key --clearmodifiers ctrl+s
215 + ;;
216 + *)
217 + echo "Unknown command: $command_name" >&2
218 + usage >&2
219 + exit 2
220 + ;;
221 +esac
tests/test_office_canvas_setup.py
+234 -7
@@ -6,7 +6,7 @@ from pathlib import Path
6 PROJECT_ROOT = Path(__file__).resolve().parents[1]
7
8
9 -def test_office_setup_canvas_uses_simple_progress_instead_of_install_logs():
9 +def test_document_canvas_uses_markdown_editor_and_official_libreoffice_desktop_frame():
10 panel = (PROJECT_ROOT / "plugins" / "_office" / "webui" / "office-panel.html").read_text(
11 encoding="utf-8",
12 )
@@ -14,11 +14,72 @@ def test_office_setup_canvas_uses_simple_progress_instead_of_install_logs():
14 encoding="utf-8",
15 )
16
17 - assert "Agent Zero Office" in panel
18 - assert "setupTitle()" in panel
19 - assert "Setup in progress" in store
17 + assert "office-rich-editor" in panel
18 + assert "office-docx-pages" in panel
19 + assert "office-desktop-frame" in panel
20 + assert "data-office-desktop-frame" in panel
21 + assert 'title="LibreOffice desktop"' not in panel
22 + assert 'aria-label="Desktop"' in panel
23 + assert "office-command-button" in panel
24 + assert "office-button-label" in panel
25 + assert "grid-template-columns: minmax(0, 1fr) auto auto auto" in panel
26 + assert "flex-wrap: nowrap" in panel
27 + assert ".modal-inner.office-modal .modal-scroll" in panel
28 + assert "office-modal-resizer" in panel
29 + assert "resize: both" not in panel
30 + assert 'tabindex="0"' in panel
31 + assert "format_underlined" in panel
32 + assert "format_align_center" in panel
33 + assert "is-native-tile" in panel
34 + assert "hasOfficialOffice()" in panel
35 + assert "office_save" in store
36 + assert "desktop_save" in store
37 + assert 'callOffice("desktop")' in store
38 + assert "ensureDesktopSession" in store
39 + assert "handleOfficialOfficeClosed" in store
40 + assert "ResizeObserver" in store
41 + assert "_desktopResizeSuspended" in store
42 + assert "_desktopResizePending" in store
43 + assert "shouldDeferDesktopResize" in store
44 + assert "right-canvas-resize-start" in store
45 + assert "right-canvas-resize-end" in store
46 + assert "isDesktopSession" in store
47 + assert "desktopFrame" in store
48 + assert "unloadDesktopFrames" in store
49 + assert "restoreDesktopFrames" in store
50 + assert "primeXpraDesktopFrame" in store
51 + assert "normalizeXpraDesktopWindow" in store
52 + assert "reloadDesktopFrame" in store
53 + assert 'result?.reload' in store
54 + assert "a0_reload" in store
55 + assert "const DESKTOP_RESIZE_DELAY_MS = 80" in store
56 + assert "requestServerResize: false" in store
57 + assert "requestRefresh: false" in store
58 + assert "options.serverResize !== true" in store
59 + assert "server_is_desktop = true" in store
60 + assert "server_resize_exact = true" in store
61 + assert "_set_decorated?.(false)" in store
62 + assert "topoffset = 0" in store
63 + assert ".undecorated" in store
64 + assert "a0-xpra-desktop-frame-css" in store
65 + assert "_desktopHeartbeatTimer" in store
66 + assert "office-modal-focus-button" in store
67 + assert "officialOfficeUrl" in store
68 + assert "hasOfficialOffice" in store
69 + assert "isOfficeSocketData" in store
70 + assert "office_command" in store
71 + assert "office_key" in store
72 + assert "office_mouse" in store
73 + assert ".uno:Bold" in store
74 + assert "nativeTilesToHtml" in store
75 + assert "editorContainsFocus" in store
76 + assert "_focusAttempts" in store
77 + assert "_nativeEventQueue" in store
78 + assert "await this.awaitNativeEvents()" in store
79 + assert "<p><br></p>" in store
80 + assert "setupTitle()" not in panel
81 + assert "Setup in progress" not in store
82 assert "office-log" not in panel
21 - assert "collabora_logs" not in store
83
84
85 def test_office_dashboard_uses_cards_and_visible_tabs_for_open_files():
@@ -33,14 +94,167 @@ def test_office_dashboard_uses_cards_and_visible_tabs_for_open_files():
94 assert "office-document-card" in panel
95 assert "openCards()" in panel
96 assert "recentCards()" in panel
97 + assert "office-editor-head" not in panel
98 assert "office-recent-row" not in panel
37 - assert "sync_open_sessions" in store
99 + assert "open_documents" in store
100
101
40 -def test_right_canvas_keeps_restored_office_surface_until_registration_finishes():
102 +def test_right_canvas_surface_is_branded_as_desktop():
103 + surface = (
104 + PROJECT_ROOT
105 + / "plugins"
106 + / "_office"
107 + / "extensions"
108 + / "webui"
109 + / "right_canvas_register_surfaces"
110 + / "register-office.js"
111 + ).read_text(encoding="utf-8")
112 + handler = (
113 + PROJECT_ROOT
114 + / "plugins"
115 + / "_office"
116 + / "extensions"
117 + / "webui"
118 + / "get_tool_message_handler"
119 + / "document-artifact-handler.js"
120 + ).read_text(encoding="utf-8")
121 +
122 + assert 'title: "Desktop"' in surface
123 + assert 'icon: "desktop_windows"' in surface
124 + assert '"Desktop"' in handler
125 +
126 +
127 +def test_official_libreoffice_desktop_route_and_packages_are_declared():
128 + routes = (PROJECT_ROOT / "helpers" / "virtual_desktop_routes.py").read_text(encoding="utf-8")
129 + primitive = (PROJECT_ROOT / "helpers" / "virtual_desktop.py").read_text(encoding="utf-8")
130 + desktop = (
131 + PROJECT_ROOT / "plugins" / "_office" / "helpers" / "libreoffice_desktop.py"
132 + ).read_text(encoding="utf-8")
133 + install = (PROJECT_ROOT / "docker" / "run" / "fs" / "ins" / "install_additional.sh").read_text(
134 + encoding="utf-8",
135 + )
136 + linux_desktop_skill = (
137 + PROJECT_ROOT / "plugins" / "_office" / "skills" / "linux-desktop" / "SKILL.md"
138 + ).read_text(encoding="utf-8")
139 + linux_desktopctl = (
140 + PROJECT_ROOT / "plugins" / "_office" / "skills" / "linux-desktop" / "scripts" / "desktopctl.sh"
141 + ).read_text(encoding="utf-8")
142 + linux_calc_helper = (
143 + PROJECT_ROOT / "plugins" / "_office" / "skills" / "linux-desktop" / "scripts" / "calc_set_cell.py"
144 + ).read_text(encoding="utf-8")
145 +
146 + assert 'Mount("/desktop"' in routes
147 + assert 'Mount("/libreoffice"' not in routes
148 + assert "http.client.HTTPConnection" in routes
149 + assert "WSConnection" in routes
150 + assert "/session/" in routes
151 + assert "resize_session" in routes
152 + assert "resize_display" in primitive
153 + assert "DEFAULT_HEIGHT = 900" in primitive
154 + assert "MAX_WIDTH = 1920" in primitive
155 + assert "MAX_HEIGHT = 1080" in primitive
156 + assert "xrandr" in primitive
157 + assert "xpra-x11" in primitive
158 + assert "xpramenu" in primitive
159 + assert "floating_menu" in primitive
160 + assert '"file_transfer": "true"' in primitive
161 + assert '"sound": "true"' in primitive
162 + assert '"printing": "true"' in primitive
163 + assert "offscreen" in primitive
164 + assert "xpra" in desktop
165 + assert "xpra-html5" in desktop
166 + assert "Xvfb" in desktop
167 + assert "xfce4-session" in desktop
168 + assert "DISPLAY_START_TIMEOUT_SECONDS" in desktop
169 + assert '"shadow"' in desktop
170 + assert "--resize-display=yes" in desktop
171 + assert "--tray=no" in desktop
172 + assert "--system-tray=no" in desktop
173 + assert "--file-transfer=yes" in desktop
174 + assert "--open-files=no" in desktop
175 + assert "--open-url=no" in desktop
176 + assert "--printing=yes" in desktop
177 + assert "--cursors=no" not in desktop
178 + assert "--audio=yes" in desktop
179 + assert "--speaker=on" in desktop
180 + assert "--microphone=off" in desktop
181 + assert "_restart_xpra_shadow(session)" not in desktop
182 + assert 'result["reload"] = True' not in desktop
183 + assert "MAX_SCREEN_WIDTH}x{MAX_SCREEN_HEIGHT}x24" in desktop
184 + assert '"-ac"' in desktop
185 + assert "SYSTEM_TITLE = \"Desktop\"" in desktop
186 + assert "title=\"Desktop\"" in desktop
187 + assert "--log-file=xpra.log" in desktop
188 + assert "virtual_desktop.session_url" in desktop
189 + assert "xsetroot" in desktop
190 + assert "BLOCKING_DIALOG_TITLES" in desktop
191 + assert "xfce4-terminal" in desktop
192 + assert "thunar" in desktop
193 + assert "Browser.desktop" in desktop
194 + assert "Files.desktop" in desktop
195 + assert "org.xfce.terminal" in desktop
196 + assert "org.xfce.settings.manager" in desktop
197 + assert "firefox-esr" not in desktop
198 + assert "xfce4-settings-manager" in desktop
199 + assert "metadata::xfce-exe-checksum" in desktop
200 + assert "DESKTOP_FOLDER_LINKS" in desktop
201 + assert "HIDDEN_XPRA_DESKTOP_ENTRIES" in desktop
202 + assert "libreoffice-gtk3" in install
203 + assert "xpra" in install
204 + assert "xpra-x11" in install
205 + assert "xpra-html5" in install
206 + assert "xfce4-session" in install
207 + assert "thunar" in install
208 + assert "libglib2.0-bin" in install
209 + assert "xfce4-terminal" in install
210 + assert "firefox-esr" not in install
211 + assert "pulseaudio" in install
212 + assert "x11-xserver-utils" in install
213 + assert "xauth" in install
214 + assert "Linux Desktop Interface" in linux_desktop_skill
215 + assert "Use the external Agent Zero Browser" in linux_desktop_skill
216 + assert "/a0/usr/workdir" in linux_desktop_skill
217 + assert "/a0/usr/projects" in linux_desktop_skill
218 + assert "desktopctl.sh" in linux_desktop_skill
219 + assert "calc-set-cell" in linux_desktop_skill
220 + assert "xdotool" in linux_desktopctl
221 + assert "agent-zero-desktop" in linux_desktopctl
222 + assert "launch_app" in linux_desktopctl
223 + assert "calc_set_cell.py" in linux_desktopctl
224 + assert "wait_for_document" in linux_calc_helper
225 + assert "document.store()" in linux_calc_helper
226 + assert "read_xlsx_cell" in linux_calc_helper
227 + assert "DisposedException" in linux_calc_helper
228 +
229 +
230 +def test_right_canvas_requires_explicit_open_and_is_absent_on_mobile():
231 canvas_store = (
232 PROJECT_ROOT / "webui" / "components" / "canvas" / "right-canvas-store.js"
233 ).read_text(encoding="utf-8")
234 + canvas_html = (
235 + PROJECT_ROOT / "webui" / "components" / "canvas" / "right-canvas.html"
236 + ).read_text(encoding="utf-8")
237 + canvas_css = (
238 + PROJECT_ROOT / "webui" / "components" / "canvas" / "right-canvas.css"
239 + ).read_text(encoding="utf-8")
240 + handler = (
241 + PROJECT_ROOT
242 + / "plugins"
243 + / "_office"
244 + / "extensions"
245 + / "webui"
246 + / "get_tool_message_handler"
247 + / "document-artifact-handler.js"
248 + ).read_text(encoding="utf-8")
249 + after_loop = (
250 + PROJECT_ROOT
251 + / "plugins"
252 + / "_office"
253 + / "extensions"
254 + / "webui"
255 + / "set_messages_after_loop"
256 + / "auto-open-document-results.js"
257 + ).read_text(encoding="utf-8")
258
259 init_registration = canvas_store.index('await callJsExtensions("right_canvas_register_surfaces", this);')
260 init_ensure = canvas_store.index("this.ensureActiveSurface();", init_registration)
@@ -51,3 +265,16 @@ def test_right_canvas_keeps_restored_office_surface_until_registration_finishes(
265
266 assert init_registration < init_ensure
267 assert register_surface < register_guard < guarded_ensure < open_surface
268 + assert "right-canvas-resize-start" in canvas_store
269 + assert "right-canvas-resize-end" in canvas_store
270 + assert "dispatchResizeEvent" in canvas_store
271 + assert "this.isOpen = false;" in canvas_store
272 + assert "wasMobileMode && this.width <= MIN_WIDTH" in canvas_store
273 + assert "if (this.isMobileMode && !surface.actionOnly)" in canvas_store
274 + assert "if (this.isMobileMode)" in canvas_store
275 + assert "shouldRender()" in canvas_store
276 + assert "$store.rightCanvas.shouldRender()" in canvas_html
277 + assert "body.right-canvas-mobile-mode .right-canvas" in canvas_css
278 + assert "display: none !important" in canvas_css
279 + assert "autoOpenOfficeCanvas" not in handler
280 + assert "requestAnimationFrame" not in after_loop