@samitouri / QOSami_Agents_Zero-1 / commits / e2f43a3f

Harden internal Browser against bot detection

Run Chromium headful through Patchright on a private Xvfb display while keeping Agent Zero's page helpers in Patchright's isolated world. Reconcile the pinned Patchright package and matching architecture-specific Chromium from Browser plugin hooks so fresh Docker images and self-updated installations converge on the same runtime.

Alessandro committed Aug 13, 2026 at 00:38 UTC e2f43a3fb8df5811f2234c26337436fe5719e618

13 files changed +460 -84

docker/run/fs/ins/install_additional.sh

@@ -99,6 +99,7 @@ DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \

99 x11-xserver-utils \

100 xdotool \

101 xauth \

102 + xvfb \

103 dbus-x11 \

104 fonts-dejavu \

105 fonts-liberation \

docker/run/fs/ins/install_playwright.sh

+2 -5

@@ -4,13 +4,10 @@ set -e

4 # activate venv

5 . "/ins/setup_venv.sh" "$@"

7 -# install playwright if not installed (should be from requirements.txt)

8 -uv pip install playwright

9 -

7 # set PW installation path to temporary Browser runtime storage

8 export PLAYWRIGHT_BROWSERS_PATH=/a0/tmp/playwright

9 mkdir -p "$PLAYWRIGHT_BROWSERS_PATH"

14 -# install chromium with dependencies

11 +# preinstall Chromium for fresh images; the Browser hook also reconciles self-updated installs

12 apt-get install -y fonts-unifont libnss3 libnspr4 libatk1.0-0 libatspi2.0-0 libxcomposite1 libxdamage1 libatk-bridge2.0-0 libcups2

16 -playwright install chromium

13 +patchright install chromium --no-shell

docs/guides/browser.md

+1 -1

@@ -234,7 +234,7 @@ See [MCP Setup](mcp-setup.md) for MCP setup.

234

235 ## Troubleshooting

236

237 -- **Browser says Playwright is missing:** Docker installs already include the browser. In local development, let Agent Zero install it on first use or preinstall it with `PLAYWRIGHT_BROWSERS_PATH=tmp/playwright playwright install chromium`.

237 +- **Browser says Chromium is missing:** Docker installs already include the browser. In local development, let Agent Zero install it on first use or preinstall it with `PLAYWRIGHT_BROWSERS_PATH=tmp/playwright patchright install chromium --no-shell`.

238 - **The Browser surface does not open automatically:** That is expected. Open the Browser surface manually or ask the agent to show it.

239 - **The Canvas does not follow the agent:** Enable **Autofocus active page** in Browser settings.

240 - **Bring Your Own Browser cannot start:** Keep A0 CLI connected, verify Browser location is **Bring Your Own Browser**, and check `/browser status` in A0 CLI.

docs/guides/troubleshooting.md

+2 -2

@@ -26,7 +26,7 @@ Refer to the [Choosing your LLMs](../setup/installation.md#installing-and-using-

26 **7. How can I make Agent Zero retain memory between sessions?**

27 Use **Settings -> Backup & Restore** and avoid mapping the entire `/a0` directory. See [How to update Agent Zero](../setup/installation.md#how-to-update-agent-zero).

29 -**8. My browser tool fails or says Playwright is missing. What now?**

29 +**8. My browser tool fails or says Chromium is missing. What now?**

31 In normal Docker installs, the Browser already includes what it needs.

@@ -35,7 +35,7 @@ browser the first time it is needed. To install it ahead of time, run this from

35 the project root after installing Python requirements:

37 ```bash

38 -PLAYWRIGHT_BROWSERS_PATH=tmp/playwright playwright install chromium

38 +PLAYWRIGHT_BROWSERS_PATH=tmp/playwright patchright install chromium --no-shell

39 ```

41 If **Bring Your Own Browser** mode fails:

docs/setup/dev-setup.md

+2 -2

@@ -68,12 +68,12 @@ Now when you select one of the python files in the project, you should see prope

69 ```bash

70 pip install -r requirements.txt

71 -PLAYWRIGHT_BROWSERS_PATH=./tmp/playwright playwright install chromium

71 +PLAYWRIGHT_BROWSERS_PATH=./tmp/playwright patchright install chromium --no-shell

72 ```

74 The first command installs Python dependencies.

76 -The second command installs full Playwright Chromium into `./tmp/playwright`,

76 +The second command installs full Patchright Chromium into `./tmp/playwright`,

77 relative to the project root. Docker images use the absolute path

78 `/a0/tmp/playwright` and ship Chromium preinstalled.

plugins/_browser/AGENTS.md

@@ -32,6 +32,9 @@

32 - Do not hardcode user-specific browser paths or secrets.

33 - Browser model-preset selection resolves omitted preset fields from `_model_config`'s global `Default` preset, not from an unrelated currently scoped model selection.

34 - Internal-browser proxy settings map directly to Playwright's persistent-context proxy option, never to Bring Your Own Browser, and changes must restart active internal runtimes.

35 +- Run internal Chromium headful through Patchright on the private virtual display; do not add user-agent or header spoofing on top of the patched driver.

36 +- Browser startup and on-demand launch must converge on the Chromium revision declared by Patchright; let its installer select the host architecture rather than hardcoding x64 or ARM downloads.

37 +- `hooks.prepare_playwright_cache()` owns reconciliation of the pinned Patchright package and Chromium binary so repository self-updates and fresh images use the same setup path.

39 ## Work Guidance

plugins/_browser/extensions/python/startup_migration/_20_browser_playwright_cache.py

+1 -1

@@ -33,7 +33,7 @@ def _start_background_cache_migration() -> threading.Thread:

34 def _migrate_cache_safely() -> None:

35 try:

36 - _log_cache_migration_result(hooks.cleanup_playwright_cache())

36 + _log_cache_migration_result(hooks.prepare_playwright_cache())

37 except Exception as exc:

38 PrintStyle.warning("Browser Playwright cache migration failed:", exc)

plugins/_browser/helpers/config.py

+1 -6

@@ -30,11 +30,6 @@ DEFAULT_MAX_OPEN_TABS = 32

30 MIN_MAX_OPEN_TABS = 1

31 HARD_MAX_OPEN_TABS = 50

32 DEFAULT_HOST_BROWSER_PRIVACY_POLICY = "allow"

33 -BASE_BROWSER_ARGS = [

34 - "--no-sandbox",

35 - "--disable-dev-shm-usage",

36 - "--disable-gpu",

37 -]

35 def _normalize_extension_paths(value: Any) -> list[str]:

@@ -388,7 +383,7 @@ def describe_browser_extensions(settings: dict[str, Any] | None) -> dict[str, An

383 def build_browser_launch_config(settings: dict[str, Any] | None) -> dict[str, Any]:

384 config = normalize_browser_config(settings)

385 extensions = describe_browser_extensions(config)

391 - args = list(BASE_BROWSER_ARGS)

386 + args: list[str] = []

387 channel: str | None = None

388 browser_mode = "chromium"

389 proxy = None

plugins/_browser/helpers/playwright.py

+142 -24

@@ -1,12 +1,20 @@

1 +import atexit

2 +import json

3 import os

4 +import re

5 +import select

6 +import shutil

7 import subprocess

8 +import sys

9 +import threading

10 +from importlib import resources

11 from pathlib import Path

13 from helpers import files

15 FULL_CHROMIUM_PATTERNS = (

8 - "chromium-*/chrome-linux/chrome",

9 - "chromium-*/chrome-win/chrome.exe",

16 + "chromium-*/chrome-linux*/chrome",

17 + "chromium-*/chrome-win*/chrome.exe",

18 )

19 PLAYWRIGHT_CACHE_ENV = "A0_BROWSER_PLAYWRIGHT_CACHE_DIR"

20 PLAYWRIGHT_CACHE_DIR = ("tmp", "playwright")

@@ -14,6 +22,10 @@ RETIRED_PLAYWRIGHT_CACHE_DIRS = (

22 ("usr", "plugins", "_browser", "playwright"),

23 ("usr", "browser", "playwright"),

24 )

25 +_INSTALL_LOCK = threading.Lock()

26 +_DISPLAY_LOCK = threading.Lock()

27 +_DISPLAY_PROCESS: subprocess.Popen | None = None

28 +_DISPLAY_NAME = ""

31 def _primary_cache_dir() -> Path:

@@ -52,33 +64,139 @@ def configure_playwright_env() -> str:

64 return cache_dir

55 -def find_playwright_binary(cache_dir: Path) -> Path | None:

56 - for pattern in FULL_CHROMIUM_PATTERNS:

57 - binary = next(cache_dir.glob(pattern), None)

58 - if binary and binary.exists():

59 - return binary

60 - return None

67 +def ensure_browser_display() -> str:

68 + global _DISPLAY_NAME, _DISPLAY_PROCESS

69 +

70 + with _DISPLAY_LOCK:

71 + if _DISPLAY_PROCESS and _DISPLAY_PROCESS.poll() is None:

72 + return _DISPLAY_NAME

73 +

74 + xvfb = shutil.which("Xvfb")

75 + if not xvfb:

76 + return ""

77 +

78 + read_fd, write_fd = os.pipe()

79 + try:

80 + process = subprocess.Popen(

81 + [

82 + xvfb,

83 + "-displayfd",

84 + str(write_fd),

85 + "-screen",

86 + "0",

87 + "1365x768x24",

88 + "+extension",

89 + "GLX",

90 + "-nolisten",

91 + "tcp",

92 + "-noreset",

93 + "-ac",

94 + ],

95 + stdin=subprocess.DEVNULL,

96 + stdout=subprocess.DEVNULL,

97 + stderr=subprocess.DEVNULL,

98 + pass_fds=(write_fd,),

99 + )

100 + except OSError:

101 + os.close(read_fd)

102 + return ""

103 + finally:

104 + os.close(write_fd)

105 +

106 + try:

107 + ready, _, _ = select.select([read_fd], [], [], 5)

108 + display_number = os.read(read_fd, 32).decode().strip() if ready else ""

109 + finally:

110 + os.close(read_fd)

111 +

112 + if not display_number.isdigit() or process.poll() is not None:

113 + _terminate_browser_display(process)

114 + return ""

115 +

116 + _DISPLAY_PROCESS = process

117 + _DISPLAY_NAME = f":{display_number}"

118 + return _DISPLAY_NAME

119 +

120 +

121 +def close_browser_display() -> None:

122 + global _DISPLAY_NAME, _DISPLAY_PROCESS

123 +

124 + with _DISPLAY_LOCK:

125 + process = _DISPLAY_PROCESS

126 + _DISPLAY_PROCESS = None

127 + _DISPLAY_NAME = ""

128 + if process:

129 + _terminate_browser_display(process)

130 +

131 +

132 +def _terminate_browser_display(process: subprocess.Popen) -> None:

133 + if process.poll() is not None:

134 + return

135 + process.terminate()

136 + try:

137 + process.wait(timeout=2)

138 + except subprocess.TimeoutExpired:

139 + process.kill()

140 + process.wait(timeout=2)

141 +

142 +

143 +def find_playwright_binary(cache_dir: Path, revision: str = "") -> Path | None:

144 + prefix = f"chromium-{revision}" if revision.isdigit() else "chromium-*"

145 + binaries = [

146 + binary

147 + for pattern in FULL_CHROMIUM_PATTERNS

148 + for binary in cache_dir.glob(pattern.replace("chromium-*", prefix))

149 + if binary.exists()

150 + ]

151 + return max(binaries, key=_chromium_revision) if binaries else None

152 +

153 +

154 +def _chromium_revision(binary: Path) -> int:

155 + match = re.search(r"chromium-(\d+)", binary.as_posix())

156 + return int(match.group(1)) if match else -1

157

158

159 def get_playwright_binary() -> Path | None:

64 - return find_playwright_binary(_primary_cache_dir())

160 + cache_dir = _primary_cache_dir()

161 + binary = find_playwright_binary(_primary_cache_dir())

162 + revision = get_playwright_chromium_revision()

163 + if revision and (not binary or _chromium_revision(binary) != int(revision)):

164 + return find_playwright_binary(cache_dir, revision=revision)

165 + return binary

166 +

167 +

168 +def get_playwright_chromium_revision() -> str:

169 + try:

170 + manifest = resources.files("patchright").joinpath("driver/package/browsers.json")

171 + browsers = json.loads(manifest.read_text(encoding="utf-8"))["browsers"]

172 + revision = next(

173 + str(browser.get("revision", ""))

174 + for browser in browsers

175 + if browser.get("name") == "chromium"

176 + )

177 + except (ImportError, FileNotFoundError, KeyError, StopIteration, TypeError, ValueError):

178 + return ""

179 + return revision if revision.isdigit() else ""

180

181

182 def ensure_playwright_binary() -> Path:

68 - binary = get_playwright_binary()

69 - if binary:

183 + with _INSTALL_LOCK:

184 + binary = get_playwright_binary()

185 + if binary:

186 + return binary

187 +

188 + cache_dir = configure_playwright_env()

189 + env = os.environ.copy()

190 + env["PLAYWRIGHT_BROWSERS_PATH"] = cache_dir

191 + subprocess.check_call(

192 + [sys.executable, "-m", "patchright", "install", "chromium", "--no-shell"],

193 + env=env,

194 + )

195 +

196 + binary = get_playwright_binary()

197 + if not binary:

198 + raise RuntimeError("Patchright Chromium binary not found after installation")

199 return binary

200

72 - cache_dir = configure_playwright_env()

73 - env = os.environ.copy()

74 - env["PLAYWRIGHT_BROWSERS_PATH"] = cache_dir

75 - install_command = ["playwright", "install", "chromium"]

76 - subprocess.check_call(

77 - install_command,

78 - env=env,

79 - )

80 -

81 - binary = get_playwright_binary()

82 - if not binary:

83 - raise RuntimeError("Playwright Chromium binary not found after installation")

84 - return binary

201 +

202 +atexit.register(close_browser_display)

plugins/_browser/helpers/runtime.py

+50 -18

@@ -27,7 +27,9 @@ from plugins._browser.helpers.config import (

27 build_browser_launch_config,

28 get_browser_config,

29 )

30 -from plugins._browser.helpers.playwright import configure_playwright_env, ensure_playwright_binary

30 +from plugins._browser.helpers.playwright import (

31 + ensure_browser_display,

32 +)

33 from plugins._browser.helpers.url import normalize_url

@@ -591,6 +593,7 @@ class _BrowserRuntimeCore:

593 self._closing = False

594 self._pending_popups: list[asyncio.Future[int]] = []

595 self._background_popup_pages: set[int] = set()

596 + self._bootstrap_page: Any | None = None

597

598 def _ensure_registry_lock(self) -> asyncio.Lock:

599 if self._registry_lock is None:

@@ -773,6 +776,7 @@ class _BrowserRuntimeCore:

776 waiter.set_exception(RuntimeError("Browser context closed."))

777 self._pending_popups.clear()

778 self._background_popup_pages.clear()

779 + self._bootstrap_page = None

780 self.pages.clear()

781 self.last_interacted_browser_id = None

782 for screencast in self.screencasts.values():

@@ -797,20 +801,26 @@ class _BrowserRuntimeCore:

801 self.playwright = None

802

803 async def _start(self) -> None:

800 - from playwright.async_api import async_playwright

804 + from plugins._browser import hooks

805 +

806 + preparation = hooks.prepare_playwright_cache()

807 + if preparation.get("errors") or not preparation.get("binary"):

808 + problem = preparation.get("errors") or "missing binary"

809 + raise RuntimeError(f"Browser setup failed: {problem}")

810 + from patchright.async_api import async_playwright

811

812 self.profile_dir.mkdir(parents=True, exist_ok=True)

813 self.downloads_dir.mkdir(parents=True, exist_ok=True)

814 self._release_orphaned_profile_singleton()

815 browser_config = get_browser_config()

816 launch_config = build_browser_launch_config(browser_config)

807 - configure_playwright_env()

808 - browser_binary = ensure_playwright_binary()

817 + browser_binary = Path(preparation["binary"])

818 + browser_display = ensure_browser_display()

819

820 self.playwright = await async_playwright().start()

821 launch_kwargs: dict[str, Any] = {

822 "user_data_dir": str(self.profile_dir),

813 - "headless": True,

823 + "headless": not bool(browser_display),

824 "accept_downloads": True,

825 "downloads_path": str(self.downloads_dir),

826 "viewport": DEFAULT_VIEWPORT,

@@ -818,6 +828,8 @@ class _BrowserRuntimeCore:

828 "no_viewport": False,

829 "args": launch_config["args"],

830 }

831 + if browser_display:

832 + launch_kwargs["env"] = {**os.environ, "DISPLAY": browser_display}

833 if launch_config["channel"]:

834 launch_kwargs["channel"] = launch_config["channel"]

835 else:

@@ -840,11 +852,12 @@ class _BrowserRuntimeCore:

852 self.context.set_default_navigation_timeout(30000)

853 self.context.on("close", self._on_context_closed)

854 self.context.on("page", self._on_new_page_sync)

843 - await self.context.add_init_script(path=str(DOM_HELPER_PATH))

844 - await self.context.add_init_script(path=str(CONTENT_HELPER_PATH))

855

856 for page in list(self.context.pages):

857 if page.url == "about:blank":

858 + if browser_display and self._bootstrap_page is None:

859 + self._bootstrap_page = page

860 + continue

861 try:

862 await page.close()

863 except Exception:

@@ -915,7 +928,10 @@ class _BrowserRuntimeCore:

928 async def open(self, url: str = "") -> dict[str, Any]:

929 await self.ensure_started()

930 self._ensure_can_open_page()

918 - page = await self.context.new_page()

931 + page = self._bootstrap_page

932 + self._bootstrap_page = None

933 + if not page or page.is_closed():

934 + page = await self.context.new_page()

935 browser_page = await self._register_page(page)

936 self.last_interacted_browser_id = browser_page.id

937 target_url = self._initial_url(url)

@@ -1296,6 +1312,7 @@ class _BrowserRuntimeCore:

1312 result = await page.evaluate(

1313 "(payload) => globalThis.__spaceBrowserPageContent__.capture(payload || null)",

1314 payload or None,

1315 + isolated_context=True,

1316 )

1317 self._maybe_promote(resolved_id)

1318 return result or {}

@@ -1308,6 +1325,7 @@ class _BrowserRuntimeCore:

1325 result = await page.evaluate(

1326 "(ref) => globalThis.__spaceBrowserPageContent__.detail(ref)",

1327 reference_id,

1328 + isolated_context=True,

1329 )

1330 self._maybe_promote(resolved_id)

1331 return result or {}

@@ -1324,6 +1342,7 @@ class _BrowserRuntimeCore:

1342 result = await page.evaluate(

1343 "(payload) => globalThis.__spaceBrowserPageContent__.annotate(payload || null)",

1344 payload or None,

1345 + isolated_context=True,

1346 )

1347 self._maybe_promote(resolved_id)

1348 return result or {}

@@ -1332,7 +1351,7 @@ class _BrowserRuntimeCore:

1351 await self.ensure_started()

1352 resolved_id = self._resolve_browser_id(browser_id)

1353 page = self._page(resolved_id)

1335 - result = await page.evaluate(str(script or "undefined"))

1354 + result = await page.evaluate(str(script or "undefined"), isolated_context=False)

1355 self._maybe_promote(resolved_id)

1356 return {"result": result, "state": await self._state(resolved_id)}

1357

@@ -1364,6 +1383,7 @@ class _BrowserRuntimeCore:

1383 box = await page.evaluate(

1384 "(ref) => globalThis.__spaceBrowserPageContent__.boundingBoxFor(ref)",

1385 reference_id,

1386 + isolated_context=True,

1387 )

1388

1389 background = focus_popup is False or (

@@ -1515,6 +1535,7 @@ class _BrowserRuntimeCore:

1535 "action": normalized_action,

1536 "text": str(text or ""),

1537 },

1538 + isolated_context=False,

1539 ) or {}

1540 except Exception as exc:

1541 clipboard_result = {

@@ -1779,6 +1800,7 @@ class _BrowserRuntimeCore:

1800 "useOffsets": bool(offset_x or offset_y),

1801 },

1802 },

1803 + isolated_context=True,

1804 )

1805 if not point or not isinstance(point, dict):

1806 raise ValueError(f"Could not resolve Browser ref {reference_id!r} to a viewport point")

@@ -1995,6 +2017,7 @@ class _BrowserRuntimeCore:

2017 "ref": ref,

2018 "values": values if values is not None else value,

2019 },

2020 + isolated_context=True,

2021 )

2022 await self._settle(page, short=True)

2023 self._maybe_promote(resolved_id)

@@ -2016,6 +2039,7 @@ class _BrowserRuntimeCore:

2039 "ref": ref,

2040 "checked": bool(checked),

2041 },

2042 + isolated_context=True,

2043 )

2044 await self._settle(page, short=True)

2045 self._maybe_promote(resolved_id)

@@ -2036,12 +2060,14 @@ class _BrowserRuntimeCore:

2060 metadata = await page.evaluate(

2061 "(ref) => globalThis.__spaceBrowserPageContent__.fileInputFor(ref)",

2062 ref,

2063 + isolated_context=True,

2064 )

2065 handle = None

2066 try:

2067 handle = await page.evaluate_handle(

2068 "(ref) => globalThis.__spaceBrowserPageContent__.fileInputElementFor(ref)",

2069 ref,

2070 + isolated_context=True,

2071 )

2072 element = handle.as_element() if handle else None

2073 if element:

@@ -2198,11 +2224,13 @@ class _BrowserRuntimeCore:

2224 action = await page.evaluate(

2225 "(args) => globalThis.__spaceBrowserPageContent__[args.method](args.ref)",

2226 {"method": helper_method, "ref": reference_id},

2227 + isolated_context=True,

2228 )

2229 else:

2230 action = await page.evaluate(

2231 "(args) => globalThis.__spaceBrowserPageContent__[args.method](args.ref, args.text)",

2232 {"method": helper_method, "ref": reference_id, "text": text},

2233 + isolated_context=True,

2234 )

2235 await self._settle(page, short=False)

2236 self._maybe_promote(resolved_id)

@@ -2215,8 +2243,8 @@ class _BrowserRuntimeCore:

2243 *,

2244 wait_until: str = "domcontentloaded",

2245 ) -> None:

2218 - from playwright.async_api import Error as PlaywrightError

2219 - from playwright.async_api import TimeoutError as PlaywrightTimeoutError

2246 + from patchright.async_api import Error as PlaywrightError

2247 + from patchright.async_api import TimeoutError as PlaywrightTimeoutError

2248

2249 try:

2250 await page.goto(url, wait_until=wait_until, timeout=30000)

@@ -2227,8 +2255,8 @@ class _BrowserRuntimeCore:

2255 await self._settle(page, short=wait_until == "commit")

2256

2257 async def _settle(self, page: Any, short: bool = False) -> None:

2230 - from playwright.async_api import Error as PlaywrightError

2231 - from playwright.async_api import TimeoutError as PlaywrightTimeoutError

2258 + from patchright.async_api import Error as PlaywrightError

2259 + from patchright.async_api import TimeoutError as PlaywrightTimeoutError

2260

2261 try:

2262 await page.wait_for_load_state(

@@ -2249,7 +2277,10 @@ class _BrowserRuntimeCore:

2277 except Exception:

2278 title = ""

2279 try:

2252 - history_length = await page.evaluate("() => globalThis.history?.length || 0")

2280 + history_length = await page.evaluate(

2281 + "() => globalThis.history?.length || 0",

2282 + isolated_context=False,

2283 + )

2284 except Exception:

2285 history_length = 0

2286 return {

@@ -2384,13 +2415,14 @@ class _BrowserRuntimeCore:

2415 async def _ensure_content_helper(self, page: Any) -> None:

2416 await self._ensure_dom_helper(page)

2417 has_helper = await page.evaluate(

2387 - "() => Boolean(globalThis.__spaceBrowserPageContent__?.ready?.())"

2418 + "() => Boolean(globalThis.__spaceBrowserPageContent__?.ready?.())",

2419 + isolated_context=True,

2420 )

2421 if has_helper:

2422 return

2423 if self._content_helper_source is None:

2424 self._content_helper_source = CONTENT_HELPER_PATH.read_text(encoding="utf-8")

2393 - await page.evaluate(self._content_helper_source)

2425 + await page.evaluate(self._content_helper_source, isolated_context=True)

2426

2427 async def _ensure_dom_helper(self, page: Any) -> None:

2428 if self._dom_helper_source is None:

@@ -2408,13 +2440,13 @@ class _BrowserRuntimeCore:

2440 targets = frames

2441 for target in targets:

2442 try:

2411 - has_helper = await target.evaluate(ready_script)

2443 + has_helper = await target.evaluate(ready_script, isolated_context=True)

2444 except Exception:

2445 continue

2446 if has_helper:

2447 continue

2448 with contextlib.suppress(Exception):

2417 - await target.evaluate(source)

2449 + await target.evaluate(source, isolated_context=True)

2450

2451 _runtimes: dict[str, BrowserRuntime] = {}

2452 _runtime_lock = threading.RLock()

plugins/_browser/hooks.py

+65

@@ -1,6 +1,12 @@

1 from __future__ import annotations

3 +import importlib

4 +import importlib.metadata

5 +import importlib.util

6 import shutil

7 +import subprocess

8 +import sys

9 +import threading

10 from pathlib import Path

12 from helpers import files, plugins, yaml as yaml_helper

@@ -10,6 +16,7 @@ from plugins._browser.helpers.config import (

16 normalize_browser_config,

17 )

18 from plugins._browser.helpers.playwright import (

19 + ensure_playwright_binary,

20 find_playwright_binary,

21 get_playwright_cache_dir,

22 get_retired_playwright_cache_dirs,

@@ -17,6 +24,11 @@ from plugins._browser.helpers.playwright import (

24 from plugins._browser.helpers.runtime import close_all_runtimes_sync

27 +_SETUP_LOCK = threading.Lock()

28 +_PLUGIN_DIR = Path(__file__).resolve().parent

29 +_ROOT_REQUIREMENTS_FILE = _PLUGIN_DIR.parents[1] / "requirements.txt"

30 +

31 +

32 def _load_saved_browser_config(project_name: str = "", agent_profile: str = "") -> dict:

33 entries = plugins.find_plugin_assets(

34 plugins.CONFIG_FILE_NAME,

@@ -95,6 +107,59 @@ def cleanup_playwright_cache() -> dict:

107 return result

108

109

110 +def prepare_playwright_cache() -> dict:

111 + with _SETUP_LOCK:

112 + _ensure_patchright_dependency()

113 + result = cleanup_playwright_cache()

114 + if result["errors"]:

115 + return result

116 + result["binary"] = str(ensure_playwright_binary())

117 + return result

118 +

119 +

120 +def install() -> dict:

121 + return prepare_playwright_cache()

122 +

123 +

124 +def _ensure_patchright_dependency() -> None:

125 + requirement = _patchright_requirement()

126 + if _patchright_is_current(requirement):

127 + return

128 +

129 + uv = shutil.which("uv")

130 + if not uv:

131 + raise RuntimeError("Browser plugin requires 'uv' to install Patchright automatically")

132 +

133 + subprocess.check_call(

134 + [uv, "pip", "install", "--python", sys.executable, requirement],

135 + cwd=str(_PLUGIN_DIR),

136 + )

137 + importlib.invalidate_caches()

138 + if not _patchright_is_current(requirement):

139 + raise RuntimeError(

140 + f"Browser dependency {requirement!r} is unavailable after installation"

141 + )

142 +

143 +

144 +def _patchright_requirement() -> str:

145 + if _ROOT_REQUIREMENTS_FILE.is_file():

146 + for line in _ROOT_REQUIREMENTS_FILE.read_text(encoding="utf-8").splitlines():

147 + requirement = line.strip()

148 + if requirement.startswith("patchright=="):

149 + return requirement

150 + raise RuntimeError(f"Browser Patchright requirement not found in {_ROOT_REQUIREMENTS_FILE}")

151 +

152 +

153 +def _patchright_is_current(requirement: str) -> bool:

154 + expected_version = requirement.partition("==")[2]

155 + if not expected_version or importlib.util.find_spec("patchright") is None:

156 + return False

157 + try:

158 + return importlib.metadata.version("patchright") == expected_version

159 + except importlib.metadata.PackageNotFoundError:

160 + return False

161 +

162 +

163 def _best_playwright_cache(candidates: list[Path]) -> Path | None:

164 valid = [path for path in candidates if path.is_dir() and find_playwright_binary(path)]

165 if not valid:

requirements.txt

+1 -1

@@ -26,7 +26,7 @@ markdown==3.7

26 mcp==1.27.0

27 newspaper3k==0.2.8

28 paramiko==3.5.0

29 -playwright==1.52.0

29 +patchright==1.61.2

30 pypdf==6.0.0

31 python-dotenv==1.1.0

32 pytz==2024.2

tests/test_browser_agent_regressions.py

+189 -24

@@ -110,6 +110,7 @@ from plugins._browser.helpers.runtime import (

110 )

111 import plugins._browser.helpers.runtime as browser_runtime_module

112 from plugins._browser.helpers.playwright import (

113 + ensure_playwright_binary,

114 get_playwright_binary,

115 get_playwright_cache_dir,

116 )

@@ -331,6 +332,9 @@ def test_browser_launch_config_uses_full_chromium_for_all_sessions(tmp_path):

332 assert default_launch["requires_full_browser"] is True

333 assert default_launch["proxy"] is None

334 assert not any(arg.startswith("--load-extension=") for arg in default_launch["args"])

335 + assert "--no-sandbox" not in default_launch["args"]

336 + assert "--disable-dev-shm-usage" not in default_launch["args"]

337 + assert "--disable-gpu" not in default_launch["args"]

338 assert "--headless=new" not in default_launch["args"]

339

340 extension_dir = tmp_path / "extension"

@@ -357,6 +361,102 @@ def _patch_playwright_cache_root(monkeypatch, tmp_path):

361 "get_abs_path",

362 lambda *parts: str(tmp_path.joinpath(*parts)),

363 )

364 + monkeypatch.setattr(

365 + browser_playwright_module,

366 + "get_playwright_chromium_revision",

367 + lambda: "1169",

368 + )

369 +

370 +

371 +def test_browser_uses_patchright_revision_when_newer_playwright_cache_exists(

372 + monkeypatch, tmp_path

373 +):

374 + _patch_playwright_cache_root(monkeypatch, tmp_path)

375 + monkeypatch.setattr(

376 + browser_playwright_module,

377 + "get_playwright_chromium_revision",

378 + lambda: "1228",

379 + )

380 + cache_dir = Path(get_playwright_cache_dir())

381 + expected = cache_dir / "chromium-1228" / "chrome-linux64" / "chrome"

382 + other = cache_dir / "chromium-1234" / "chrome-linux64" / "chrome"

383 + expected.parent.mkdir(parents=True)

384 + other.parent.mkdir(parents=True)

385 + expected.touch()

386 + other.touch()

387 +

388 + assert get_playwright_binary() == expected

389 +

390 +

391 +@pytest.mark.parametrize("platform_dir", ["chrome-linux", "chrome-linux64"])

392 +def test_browser_installs_current_patchright_chromium_for_host_architecture(

393 + monkeypatch, tmp_path, platform_dir

394 +):

395 + _patch_playwright_cache_root(monkeypatch, tmp_path)

396 + monkeypatch.setattr(

397 + browser_playwright_module,

398 + "get_playwright_chromium_revision",

399 + lambda: "1234",

400 + )

401 + old_binary = (

402 + tmp_path / "tmp" / "playwright" / "chromium-1169" / "chrome-linux" / "chrome"

403 + )

404 + old_binary.parent.mkdir(parents=True)

405 + old_binary.touch()

406 + expected = (

407 + tmp_path / "tmp" / "playwright" / "chromium-1234" / platform_dir / "chrome"

408 + )

409 +

410 + def install(command, *, env):

411 + assert command == [

412 + sys.executable,

413 + "-m",

414 + "patchright",

415 + "install",

416 + "chromium",

417 + "--no-shell",

418 + ]

419 + assert env["PLAYWRIGHT_BROWSERS_PATH"] == str(tmp_path / "tmp" / "playwright")

420 + expected.parent.mkdir(parents=True)

421 + expected.touch()

422 +

423 + monkeypatch.setattr(browser_playwright_module.subprocess, "check_call", install)

424 +

425 + assert ensure_playwright_binary() == expected

426 +

427 +

428 +def test_browser_hook_installs_patchright_for_existing_self_updated_runtime(monkeypatch):

429 + current = False

430 +

431 + def is_current(requirement):

432 + assert requirement == "patchright==1.61.2"

433 + return current

434 +

435 + def install(command, *, cwd):

436 + nonlocal current

437 + assert command == [

438 + "/usr/local/bin/uv",

439 + "pip",

440 + "install",

441 + "--python",

442 + sys.executable,

443 + "patchright==1.61.2",

444 + ]

445 + assert cwd == str(PROJECT_ROOT / "plugins" / "_browser")

446 + current = True

447 +

448 + monkeypatch.setattr(

449 + browser_hooks_module,

450 + "_patchright_requirement",

451 + lambda: "patchright==1.61.2",

452 + )

453 + monkeypatch.setattr(browser_hooks_module, "_patchright_is_current", is_current)

454 + monkeypatch.setattr(browser_hooks_module.shutil, "which", lambda name: "/usr/local/bin/uv")

455 + monkeypatch.setattr(browser_hooks_module.subprocess, "check_call", install)

456 +

457 + browser_hooks_module._ensure_patchright_dependency()

458 +

459 + assert current is True

460

461

462 def _write_playwright_binary(cache_dir: Path) -> Path:

@@ -1705,23 +1805,18 @@ def test_browser_runtime_requires_current_content_helper_for_modifier_clicks():

1805 ).read_text(encoding="utf-8")

1806

1807 assert "__spaceBrowserPageContent__?.ready?.()" in runtime

1808 + assert "context.add_init_script" not in runtime

1809 + assert "isolated_context=False" in runtime

1810

1811

1812 @pytest.mark.anyio

1813 async def test_browser_dom_helper_clicks_content_ref_inside_iframe():

1712 - pytest.importorskip("playwright.async_api")

1713 - from playwright.async_api import async_playwright

1814 + pytest.importorskip("patchright.async_api")

1815 + from patchright.async_api import async_playwright

1816

1817 browser_binary = get_playwright_binary()

1818 if not browser_binary:

1717 - pytest.skip("Playwright Chromium binary is not installed")

1718 -

1719 - dom_helper = (

1720 - PROJECT_ROOT / "plugins" / "_browser" / "assets" / "browser-dom-helper.js"

1721 - ).read_text(encoding="utf-8")

1722 - content_helper = (

1723 - PROJECT_ROOT / "plugins" / "_browser" / "assets" / "browser-page-content.js"

1724 - ).read_text(encoding="utf-8")

1819 + pytest.skip("Patchright Chromium binary is not installed")

1820

1821 async with async_playwright() as playwright:

1822 try:

@@ -1731,12 +1826,10 @@ async def test_browser_dom_helper_clicks_content_ref_inside_iframe():

1826 args=["--no-sandbox"],

1827 )

1828 except Exception as exc:

1734 - pytest.skip(f"Playwright Chromium could not launch: {exc}")

1829 + pytest.skip(f"Patchright Chromium could not launch: {exc}")

1830

1831 try:

1832 context = await browser.new_context()

1738 - await context.add_init_script(dom_helper)

1739 - await context.add_init_script(content_helper)

1833 page = await context.new_page()

1834 await page.set_content(

1835 """

@@ -1755,9 +1848,8 @@ async def test_browser_dom_helper_clicks_content_ref_inside_iframe():

1848 </html>

1849 """

1850 )

1758 - await page.wait_for_function(

1759 - "() => Boolean(document.querySelector('iframe')?.contentWindow?.__spaceBrowserDomHelper__)"

1760 - )

1851 + core = _BrowserRuntimeCore("patchright-helper")

1852 + await core._ensure_content_helper(page)

1853

1854 captured = await page.evaluate(

1855 "(payload) => globalThis.__spaceBrowserPageContent__.capture(payload || null)",

@@ -2093,13 +2185,32 @@ def test_browser_docker_installs_full_chromium_to_tmp_cache():

2185 script = (

2186 PROJECT_ROOT / "docker" / "run" / "fs" / "ins" / "install_playwright.sh"

2187 ).read_text(encoding="utf-8")

2188 + requirements = (PROJECT_ROOT / "requirements.txt").read_text(encoding="utf-8")

2189

2190 assert "PLAYWRIGHT_BROWSERS_PATH=/a0/tmp/playwright" in script

2098 - assert "playwright install chromium" in script

2191 + assert "patchright install chromium --no-shell" in script

2192 + assert "playwright install chromium" not in script

2193 + assert "uv pip install" not in script

2194 + assert "patchright==1.61.2" in requirements

2195 assert "--only-shell" not in script

2196

2197 + runtime = (PROJECT_ROOT / "plugins" / "_browser" / "helpers" / "runtime.py").read_text(

2198 + encoding="utf-8"

2199 + )

2200 + assert "from patchright.async_api import async_playwright" in runtime

2201 + assert "from playwright.async_api import async_playwright" not in runtime

2202 + assert runtime.index("hooks.prepare_playwright_cache()") < runtime.index(

2203 + "from patchright.async_api import async_playwright"

2204 + )

2205 + install_additional = (

2206 + PROJECT_ROOT / "docker" / "run" / "fs" / "ins" / "install_additional.sh"

2207 + ).read_text(encoding="utf-8")

2208 + assert '"headless": not bool(browser_display)' in runtime

2209 + assert 'launch_kwargs["env"] = {**os.environ, "DISPLAY": browser_display}' in runtime

2210 + assert " xvfb \\" in install_additional

2211 +

2212

2102 -def test_browser_startup_migration_runs_playwright_cache_cleanup():

2213 +def test_browser_startup_migration_prepares_current_playwright_binary():

2214 extension = (

2215 PROJECT_ROOT

2216 / "plugins"

@@ -2111,7 +2222,7 @@ def test_browser_startup_migration_runs_playwright_cache_cleanup():

2222 ).read_text(encoding="utf-8")

2223

2224 assert "class BrowserPlaywrightCacheMigration(Extension)" in extension

2114 - assert "hooks.cleanup_playwright_cache()" in extension

2225 + assert "hooks.prepare_playwright_cache()" in extension

2226 assert "PrintStyle.warning" in extension

2227

2228

@@ -2135,6 +2246,55 @@ def test_browser_runtime_removes_stale_profile_singletons(monkeypatch, tmp_path)

2246 )

2247

2248

2249 +@pytest.mark.anyio

2250 +async def test_browser_first_open_reuses_headful_bootstrap_page(monkeypatch):

2251 + class BootstrapPage:

2252 + url = "about:blank"

2253 +

2254 + @staticmethod

2255 + def is_closed():

2256 + return False

2257 +

2258 + page = BootstrapPage()

2259 +

2260 + class Context:

2261 + pages = [page]

2262 +

2263 + @staticmethod

2264 + async def new_page():

2265 + raise AssertionError("The first headful tab must reuse Chrome's bootstrap page")

2266 +

2267 + core = _BrowserRuntimeCore("headful")

2268 + core.context = Context()

2269 + core._bootstrap_page = page

2270 +

2271 + async def register(registered_page):

2272 + assert registered_page is page

2273 + browser_page = BrowserPage(id=1, page=registered_page)

2274 + core.pages[1] = browser_page

2275 + return browser_page

2276 +

2277 + async def settle(_page, short=False):

2278 + return None

2279 +

2280 + async def state(browser_id):

2281 + return {"id": browser_id, "currentUrl": "about:blank"}

2282 +

2283 + core._register_page = register

2284 + core._settle = settle

2285 + core._state = state

2286 + monkeypatch.setattr(

2287 + browser_runtime_module,

2288 + "get_browser_config",

2289 + lambda: {"default_homepage": "about:blank", "max_open_tabs": 8},

2290 + )

2291 +

2292 + result = await core.open()

2293 +

2294 + assert result == {"id": 1, "state": {"id": 1, "currentUrl": "about:blank"}}

2295 + assert core._bootstrap_page is None

2296 +

2297 +

2298 @pytest.mark.anyio

2299 async def test_browser_runtime_restarts_when_cached_context_is_stale():

2300 starts = []

@@ -2780,7 +2940,12 @@ async def test_browser_viewer_subscribe_returns_initial_snapshot(monkeypatch):

2940

2941 result = await handler.process(

2942 "browser_viewer_subscribe",

2783 - {"context_id": "ctx", "browser_id": 1, "viewport_width": 900, "viewport_height": 600},

2943 + {

2944 + "context_id": "ctx",

2945 + "browser_id": 1,

2946 + "viewport_width": 900,

2947 + "viewport_height": 600,

2948 + },

2949 "sid-snapshot",

2950 )

2951

@@ -3202,7 +3367,7 @@ async def test_browser_runtime_screenshot_file_defaults_to_chat_scoped_artifact(

3367 async def title(self):

3368 return "Blank"

3369

3205 - async def evaluate(self, script, payload=None):

3370 + async def evaluate(self, script, payload=None, **kwargs):

3371 return 1

3372

3373 core = _BrowserRuntimeCore("ctx/id")

@@ -3321,7 +3486,7 @@ async def test_browser_runtime_ref_point_resolution_applies_offsets():

3486 def __init__(self):

3487 self.mouse = FakeMouse()

3488

3324 - async def evaluate(self, script, payload=None):

3489 + async def evaluate(self, script, payload=None, **kwargs):

3490 eval_payloads.append((script, payload))

3491 if payload and "offsets" in payload:

3492 return {

@@ -3392,7 +3557,7 @@ async def test_browser_runtime_clipboard_paste_uses_dom_bridge():

3557 def __init__(self):

3558 self.keyboard = FakeKeyboard()

3559

3395 - async def evaluate(self, script, payload=None):

3560 + async def evaluate(self, script, payload=None, **kwargs):

3561 if payload is not None:

3562 eval_payloads.append((script, payload))

3563 return {

@@ -3442,7 +3607,7 @@ async def test_browser_runtime_clipboard_paste_falls_back_to_keyboard_insert_tex

3607 def __init__(self):

3608 self.keyboard = FakeKeyboard()

3609

3445 - async def evaluate(self, script, payload=None):

3610 + async def evaluate(self, script, payload=None, **kwargs):

3611 if payload is not None:

3612 return {

3613 "action": "paste",