@samitouri / QOSami_Agents_Zero-1 / commits / d0c5e62f

Added _vision_sidecar

GreifMax committed Aug 20, 2026 at 00:43 UTC d0c5e62f2357ab053a9de4ffe587a2d9b82c6ddd
16 files changed +1420
plugins/_vision_sidecar/LICENSE new
+21
@@ -0,0 +1,21 @@
1 +MIT License
2 +
3 +Copyright (c) 2026 Vision Sidecar contributors
4 +
5 +Permission is hereby granted, free of charge, to any person obtaining a copy
6 +of this software and associated documentation files (the "Software"), to deal
7 +in the Software without restriction, including without limitation the rights
8 +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 +copies of the Software, and to permit persons to whom the Software is
10 +furnished to do so, subject to the following conditions:
11 +
12 +The above copyright notice and this permission notice shall be included in all
13 +copies or substantial portions of the Software.
14 +
15 +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 +SOFTWARE.
plugins/_vision_sidecar/README.md new
+154
@@ -0,0 +1,154 @@
1 +# Vision Sidecar
2 +
3 +![Vision Sidecar](webui/thumbnail.jpg)
4 +
5 +Tolerant `vision_load` + optional dedicated Vision Model.
6 +
7 +## Why
8 +
9 +Two recurring pain points in Agent Zero:
10 +
11 +1. **Bare-string bug.** Core `vision_load` requires `paths` as a list. A bare string `"/a.png"` is iterated char-by-char, loads 0 images, and wastes a turn with no error.
12 +2. **No vision, no images.** Frontier reasoners (GLM 5.2/5.3, DeepSeek V4 Flash/Pro) are cheap and strong but have no vision. On stock A0 that means no `vision_load` at all — even though a cheap `gpt-4o-mini` or `qwen2-vl` could read the image for pennies.
13 +
14 +Vision Sidecar fixes both in one plugin.
15 +
16 +## What it does
17 +
18 +### 1. Tolerant `vision_load`
19 +
20 +`vision_load` now accepts `paths` as `string` or `list[str]`.
21 +
22 +- `{"paths": "/a.png"}` is treated as `["/a.png"]`
23 +- Handles harness quirks: JSON-encoded array strings (`"[\"/a.png\"]"`), quoted single paths (`"\"/a.png\""`)
24 +- Wrong types return a clear tool error — never a `Message misformat`
25 +
26 +### 2. Delegated Vision Model
27 +
28 +Configure an optional **Vision Model** in **Settings → Model Presets → Vision Model**.
29 +
30 +> Optional dedicated model for vision_load — used when Main has no vision. Leave empty to use Main's vision.
31 +
32 +When set:
33 +
34 +- `vision_load(paths, query?, raw?)` materializes images, calls the Vision Model with `query + images`, and returns a **text capsule** instead of injecting `~1500 tok/image` into the main history.
35 +- Your Main (GLM, DeepSeek) never sees raw pixels — only ~300 tokens of focused text. Saves thousands of tokens per future turn.
36 +- `query` is a focused instruction: `"read the top-right error toast"`, `"locate the login button and give coordinates"`. Empty → generic precise description.
37 +- `raw=true` bypasses delegation and injects images directly into Main. Use for side-by-side comparison when Main must see pixels.
38 +- Large images over ~900 KB are auto-compressed to 1280×960 JPEG before the vision call to avoid `Request Entity Too Large` (4 MB PNG → ~250 KB).
39 +
40 +When empty: legacy path — images are injected as `RawMessage` for `chat_model.vision == true`, appearance identical to stock A0.
41 +
42 +Preset defaults for the Vision slot: **64000 context, 70% for history** (new presets only). Existing presets are untouched.
43 +
44 +## Requirements
45 +
46 +- Agent Zero. If your **Settings → Model Presets → Edit** already shows **Main / Vision / Utility / Embedding** (only if you're updating the plugin), nothing else to do.
47 +- If it only shows **Main / Utility / Embedding** (on any A0 instance), run the one-time Vision-slot patch below — otherwise Vision Sidecar still works, but `vision_load` falls back to tolerant direct injection (no delegation).
48 +- Any LiteLLM-compatible vision model for the Vision slot (tested with `openai/gpt-4o-mini`, `qwen2-vl`).
49 +
50 +## Installation
51 +
52 +### From ZIP
53 +
54 +1. Download `vision_sidecar.zip` from Releases
55 +2. Agent Zero → **Settings → Plugins → Install → From ZIP** → select the ZIP
56 +3. Add the Vision slot via the script (check below)
57 +4. Restart the WebUI (`Ctrl+Shift+R`)
58 +
59 +### From Git
60 +
61 +```bash
62 +git clone https://github.com/GreifMax/a0-vision-sidecar
63 +cp -r a0-vision_sidecar /a0/usr/plugins/vision_sidecar
64 +# restart Agent Zero
65 +```
66 +
67 +### Add Vision slot
68 +
69 +Since **v0.4.0** the Vision slot is applied **automatically on install** (via the plugin's `install()` hook) — no manual step. If Model Presets still shows only Main / Utility / Embedding (e.g. after an A0 core update overwrote `plugins/_model_config`), re-run it in one click:
70 +
71 +- **Settings → Plugins → Vision Sidecar → Execute** (preferred), or
72 +- the manual script below (same logic, also usable for `--status` / `--restore`)
73 +
74 +The patcher is **self-contained pure Python** — no git or `patch(1)` required, idempotent, and creates `.vision_sidecar.bak` backups of every modified file.
75 +
76 +```bash
77 +# any directory works; it auto-finds the Agent Zero root
78 +bash /a0/usr/plugins/vision_sidecar/scripts/enable_vision_slot.sh
79 +# or: python3 /a0/usr/plugins/vision_sidecar/scripts/enable_vision_slot.py
80 +```
81 +
82 +Docker (run **inside** the Agent Zero container, not on the host — `plugins/` only exists in the image):
83 +
84 +```bash
85 +docker exec -it <agent-zero-container> bash /a0/usr/plugins/vision_sidecar/scripts/enable_vision_slot.sh
86 +```
87 +
88 +Options:
89 +
90 +| Command | Effect |
91 +| --- | --- |
92 +| (no args) | Apply patch (skips files already patched) |
93 +| `--status` | Show per-file state without changing anything |
94 +| `--restore` | Restore all original files from `.bak` backups |
95 +
96 +If auto-detection fails, point it at your install: `A0_ROOT=/path/to/agent-zero bash enable_vision_slot.sh`.
97 +
98 +Then restart Agent Zero and hard-refresh the browser (Ctrl+Shift+R). Model Presets will show **Main / Vision / Utility / Embedding**.
99 +
100 +> Note: A0 updates can overwrite `plugins/_model_config`. After updating, rerun the script — it is idempotent and will re-apply cleanly.
101 +
102 +## Configuration
103 +
104 +1. **Settings → Model Presets → Edit** → fill **Vision Model** with your cheap vision helper (provider + name + key). Leave empty to use Main's vision.
105 +2. **Main Model → Supports Vision on** → optional **Overrides Vision Model** switch appears right under it: when on, Main's native vision is always used for that preset and the Vision Model is ignored; when off (default), the dedicated Vision Model handles vision when configured. The chat model switcher hides the Vision row for presets where the override is on, and the Agent Config preset preview shows "Overwritten by Main" in place of the Vision model when the override is active.
106 +3. **Settings → Plugins → Vision Sidecar** → tune the delegated system prompt and timeout if needed.
107 +
108 +New presets automatically get `Vision: 64000 / 0.7`. Current presets keep their values. The override flag is per-preset (never inherited from Default).
109 +
110 +## Usage (By A0)
111 +
112 +```json
113 +{
114 + "tool_name": "vision_load",
115 + "tool_args": {
116 + "paths": ["/a0/usr/uploads/screenshot.png"],
117 + "query": "read the error message in the top-right"
118 + }
119 +}
120 +```
121 +
122 +- With Vision Model set -> chat shows thumbnails + `N images sent, M images skipped - Description: "..."` (counts + vision-model capsule in one line). The tool step always includes a Query row (Paths / Tool Name / Query / Result), even when the call omitted `query`.
123 +- The delegated `vision_load` prompt declares an explicit JSON tool schema (`paths`, `query`, `raw`), so models see `query` as a real parameter. When the Main model overrides vision (or there is no Vision Model), the stock prompt is used and `query` is absent from the schema. The schema rejects unknown properties (`additionalProperties: false`), and the tool normalizes prompt-style aliases (`Prompt`, `question`, `instruction`, ...) into `query`, so the vision model always receives the intended focus text even if a model ignores the schema.
124 +- With `raw=true` → forces direct injection even when Vision Model is set.
125 +- Without Vision Model -> fully stock: stock prompt (no `query`/`raw`), `Loaded images: N` with thumbnails for Main vision, and no vision tool at all when Main has no vision.
126 +- With **Overrides Vision Model** on (Main vision-capable presets) -> Main's native vision is used even though a Vision Model is set; delegation (and the switcher's Vision row) is skipped for that preset.
127 +
128 +## Reliability notes
129 +
130 +- `vision_load` is safe inside the `parallel` tool: the job result is the real text capsule (or error), never a placeholder — the tool sets the authoritative response message in every outcome path.
131 +- Image blocks are injected into main history only when the preset's main model declares vision support (delegated and legacy paths alike). With a text-only main, delegated calls return the text capsule only and `raw=true` auto-delegates.
132 +- The `vision` flag is the user's declaration and governs: if it mislabels a text-only provider, the provider may reject image blocks (`400 content.type invalid`) — fix the flag in Model Presets; the text capsule still carries the answer meanwhile.
133 +- Inside `parallel` workers the tool writes the `Result` row of its log item. The parent job aggregator skips the body text write when the Result row already carries the same text, so the step shows the text only once (inside the Result row). Errors and tools without a Result row still get body text.
134 +- With no Vision Model configured and a text-only main, the tool reports loaded/skipped counts plus an explicit note that images were not injected.
135 +
136 +## File layout
137 +
138 +```
139 +plugin.yaml
140 +default_config.yaml
141 +LICENSE
142 +README.md
143 +thumbnail.jpg ← plugin list image (256×256, ≤20 KB)
144 +helpers/vision_model.py ← preset-aware vision dispatch + compression
145 +tools/vision_load.py ← tolerant paths + delegation
146 +prompts/agent.system.tool.vision_load.md
147 +extensions/python/system_prompt/_10_vision_sidecar_guidance.py
148 +webui/config.html
149 +webui/thumbnail.jpg/png
150 +```
151 +
152 +## License
153 +
154 +MIT — see [LICENSE](LICENSE).
plugins/_vision_sidecar/default_config.yaml new
+4
@@ -0,0 +1,4 @@
1 +behaviour:
2 + delegated_system: "You are a precise vision analyst. Answer only what was asked about the image(s). Be concise, factual, mention positions/coordinates when asked to locate."
3 + max_tokens: 2000
4 + timeout: 300
plugins/_vision_sidecar/execute.py new
+28
@@ -0,0 +1,28 @@
1 +"""Vision Sidecar — manual maintenance script.
2 +
3 +Triggered from Settings → Plugins → Vision Sidecar (Execute). Useful after an
4 +A0 update that overwrote plugins/_model_config: re-applies the Vision-slot
5 +patch idempotently (already-patched files are skipped). The install hook does
6 +this automatically on plugin install, so most users never need this.
7 +"""
8 +import sys
9 +from pathlib import Path
10 +
11 +
12 +def main() -> int:
13 + try:
14 + hooks = Path(__file__).resolve().parent / "hooks.py"
15 + import importlib.util
16 + spec = importlib.util.spec_from_file_location("vision_sidecar_hooks", hooks)
17 + mod = importlib.util.module_from_spec(spec)
18 + spec.loader.exec_module(mod)
19 + mod.install()
20 + print("Vision Sidecar maintenance completed successfully.")
21 + return 0
22 + except Exception as e:
23 + print(f"ERROR: {e}")
24 + return 1
25 +
26 +
27 +if __name__ == "__main__":
28 + sys.exit(main())
plugins/_vision_sidecar/execute_record.json new
+1
@@ -0,0 +1 @@
1 +{"executed_at": "2026-08-19T18:55:52.671678+02:00", "exit_code": 0}
\ No newline at end of file
plugins/_vision_sidecar/extensions/python/system_prompt/_10_vision_sidecar_guidance.py new
+21
@@ -0,0 +1,21 @@
1 +from helpers.extension import Extension
2 +
3 +_D = "Vision Sidecar active: a dedicated Vision Model IS configured (Model Presets -> Vision Model). vision_load DELEGATES:"
4 +_D += chr(10) + "- send a focused query about the images (e.g. read the top-right error toast, locate the login button and give its position) - you will get a concise text capsule, not pixels."
5 +_D += chr(10) + "- if Main cannot see images, ~1500 tok/image stays out of your context (capsule only); if Main is vision-capable, the images are also attached to history alongside the capsule. Use raw=true only when you need pixels without a capsule (e.g. side-by-side comparison)."
6 +_GUIDANCE_DELEGATED = _D
7 +
8 +
9 +class VisionSidecarGuidance(Extension):
10 + async def execute(self, system_prompt: list[str] | None = None, **kwargs):
11 + # With no dedicated Vision Model the plugin is fully stock: no guidance,
12 + # stock prompt, stock result format.
13 + if system_prompt is None:
14 + return
15 + try:
16 + from usr.plugins.vision_sidecar.helpers.vision_model import has_vision_model
17 + if not has_vision_model(self.agent):
18 + return
19 + except Exception:
20 + return
21 + system_prompt.append(_GUIDANCE_DELEGATED)
plugins/_vision_sidecar/helpers/vision_model.py new
+128
@@ -0,0 +1,128 @@
1 +from __future__ import annotations
2 +
3 +from typing import Any
4 +
5 +import models
6 +
7 +DEFAULT_DELEGATED_SYSTEM = (
8 + "You are a precise vision analyst. Answer only what was asked about the image(s). "
9 + "Be concise, factual, mention positions/coordinates when asked to locate."
10 +)
11 +
12 +def get_behaviour(agent: Any = None) -> dict[str, Any]:
13 + from helpers import plugins
14 + cfg = plugins.get_plugin_config("vision_sidecar", agent=agent) or {}
15 + b = cfg.get("behaviour") or {}
16 + if not isinstance(b, dict):
17 + b = {}
18 + return {
19 + "delegated_system": str(b.get("delegated_system") or DEFAULT_DELEGATED_SYSTEM),
20 + "max_tokens": int(b.get("max_tokens") or 2000),
21 + "timeout": float(b.get("timeout") or 300),
22 + }
23 +
24 +def get_vision_model_config(agent: Any = None) -> dict[str, Any]:
25 + """Read Vision Model from Model Presets (vision slot), not from sidecar config.
26 +
27 + Preset storage is in _model_config plugin: PRESET_SLOT_CONFIG_SECTIONS["vision"] = "vision_model".
28 + Uses get_effective_config so per-chat overrides are respected.
29 + """
30 + try:
31 + from plugins._model_config.helpers.model_config import get_effective_config
32 + cfg = get_effective_config(agent) or {}
33 + vm = cfg.get("vision_model") or {}
34 + if not isinstance(vm, dict):
35 + return {}
36 + provider = str(vm.get("provider") or "").strip()
37 + name = str(vm.get("name") or "").strip()
38 + if not provider and not name:
39 + return {}
40 + return vm
41 + except Exception:
42 + return {}
43 +
44 +def has_vision_model(agent: Any = None) -> bool:
45 + vm = get_vision_model_config(agent)
46 + if not (vm.get("provider") and vm.get("name")):
47 + return False
48 + try:
49 + from plugins._model_config.helpers.model_config import get_chat_model_config
50 + chat_cfg = get_chat_model_config(agent) or {}
51 + if chat_cfg.get("vision", False) and chat_cfg.get("vision_override", False):
52 + return False # Main's native vision overrides the Vision Model
53 + except Exception:
54 + pass
55 + return True
56 +
57 +def build_vision_model(agent: Any = None):
58 + vm = get_vision_model_config(agent)
59 + if not vm:
60 + return None
61 + from plugins._model_config.helpers.model_config import build_model_config
62 + mc = build_model_config(vm, models.ModelType.CHAT)
63 + mc.vision = True
64 + return models.get_chat_model(mc.provider, mc.name, model_config=mc, **mc.build_kwargs())
65 +
66 +async def call_vision_model(
67 + agent: Any,
68 + images_a0_paths: list[str],
69 + query: str,
70 + delegated_system: str | None = None,
71 + timeout: float = 300,
72 +) -> str:
73 + """Call the dedicated vision preset model with images + query, return text capsule."""
74 + import asyncio
75 + from langchain_core.messages import HumanMessage, SystemMessage
76 +
77 + model = build_vision_model(agent)
78 + if model is None:
79 + raise RuntimeError("vision_model not configured — set it in Model Presets → Vision Model")
80 +
81 + behaviour = get_behaviour(agent)
82 + system = (delegated_system or behaviour["delegated_system"]).strip() or DEFAULT_DELEGATED_SYSTEM
83 + if query and query.strip():
84 + user_text = query.strip()
85 + else:
86 + user_text = "Describe the image(s) precisely. Be concise, mention key objects, text, and layout."
87 +
88 + content: list[dict[str, Any]] = [{"type": "text", "text": user_text}]
89 + for pa in images_a0_paths:
90 + url = pa
91 + try:
92 + import base64
93 + from pathlib import Path as _Path
94 + from helpers import files as _files
95 + from helpers.images import compress_image as _compress
96 + raw = str(pa or "").strip()
97 + cand = None
98 + if raw.startswith("/a0/"):
99 + cand = _Path(_files.fix_dev_path(raw) if hasattr(_files, "fix_dev_path") else raw)
100 + if not cand.exists():
101 + cand = _Path(raw)
102 + else:
103 + cand = _Path(raw)
104 + if cand and cand.exists() and cand.is_file():
105 + data = cand.read_bytes()
106 + if len(data) > 900 * 1024:
107 + try:
108 + c = _compress(data, max_pixels=1280*960, quality=80)
109 + b64 = base64.b64encode(c).decode()
110 + url = f"data:image/jpeg;base64,{b64}"
111 + except Exception:
112 + url = pa
113 + else:
114 + url = pa
115 + except Exception:
116 + url = pa
117 + content.append({"type": "image_url", "image_url": {"url": url}})
118 +
119 + messages = [SystemMessage(content=system), HumanMessage(content=content)]
120 +
121 + async def _call():
122 + kwargs = {"max_tokens": int(behaviour["max_tokens"])}
123 + resp, _reason = await model.unified_call(messages=messages, explicit_caching=False, **kwargs)
124 + return resp
125 + try:
126 + return await asyncio.wait_for(_call(), timeout=float(timeout or behaviour["timeout"]))
127 + except asyncio.TimeoutError:
128 + raise TimeoutError(f"vision_model timed out after {timeout}s")
plugins/_vision_sidecar/hooks.py new
+72
@@ -0,0 +1,72 @@
1 +"""Vision Sidecar lifecycle hooks.
2 +
3 +install() - auto-applies the Vision-slot patch when the plugin is installed
4 + via the Plugins UI ("Install from ZIP/Git"). No manual script
5 + needed anymore; scripts/enable_vision_slot.py remains as a
6 + fallback/debug tool.
7 +pre_update() - re-applies the patch idempotently (A0 updates may overwrite
8 + plugins/_model_config).
9 +uninstall() - restores the original _model_config files from .bak backups,
10 + but only if the current files still contain our edits (never
11 + clobbers a core update that already replaced them).
12 +"""
13 +from __future__ import annotations
14 +
15 +import importlib.util
16 +from pathlib import Path
17 +
18 +_HOOKS_CONTEXT: dict = {}
19 +
20 +
21 +def _run_patcher(**kwargs) -> tuple[bool, str]:
22 + """Import and run scripts/enable_vision_slot.py::run(). Returns (ok, log)."""
23 + plugin_dir = Path(__file__).resolve().parent
24 + patcher = plugin_dir / "scripts" / "enable_vision_slot.py"
25 + spec = importlib.util.spec_from_file_location("vision_sidecar_patcher", patcher)
26 + mod = importlib.util.module_from_spec(spec)
27 + spec.loader.exec_module(mod)
28 + lines: list[str] = []
29 + import contextlib, io
30 + buf = io.StringIO()
31 + with contextlib.redirect_stdout(buf):
32 + rc = mod.run(**kwargs) # type: ignore[attr-defined]
33 + lines = [l for l in buf.getvalue().splitlines() if l.strip()]
34 + return (rc == 0), "\n".join(lines)
35 +
36 +
37 +def install(hook_context: dict | None = None, **kwargs):
38 + try:
39 + ok, log = _run_patcher()
40 + print("[vision_sidecar] install:")
41 + print(log)
42 + if not ok:
43 + print(
44 + "[vision_sidecar] Vision-slot patch could not be fully applied. "
45 + "vision_load still works (tolerant, direct injection); run "
46 + "scripts/enable_vision_slot.py manually to retry, or report at "
47 + "https://github.com/GreifMax/a0-vision-sidecar/issues"
48 + )
49 + except Exception as e: # never break the installer
50 + print(f"[vision_sidecar] install hook skipped: {e}")
51 + _HOOKS_CONTEXT["installed"] = True
52 + return None
53 +
54 +
55 +def pre_update(hook_context: dict | None = None, **kwargs):
56 + try:
57 + ok, log = _run_patcher()
58 + print("[vision_sidecar] pre_update:")
59 + print(log)
60 + except Exception as e:
61 + print(f"[vision_sidecar] pre_update hook skipped: {e}")
62 + return None
63 +
64 +
65 +def uninstall(hook_context: dict | None = None, **kwargs):
66 + try:
67 + ok, log = _run_patcher(restore=True)
68 + print("[vision_sidecar] uninstall:")
69 + print(log)
70 + except Exception as e:
71 + print(f"[vision_sidecar] uninstall hook skipped: {e}")
72 + return None
plugins/_vision_sidecar/plugin.yaml new
+8
@@ -0,0 +1,8 @@
1 +name: vision_sidecar
2 +title: Vision Sidecar
3 +description: Tolerant vision_load (accepts single string) + optional dedicated vision model that answers a focused query about images without bloating the main context.
4 +version: 0.7.9
5 +settings_sections:
6 + - agent
7 +per_project_config: true
8 +per_agent_config: true
plugins/_vision_sidecar/prompts/vision_sidecar.delegated.md new
+50
@@ -0,0 +1,50 @@
1 +## multimodal vision tools
2 +
3 +### vision_load
4 +load images into the model for visual reasoning via the dedicated vision model
5 +args: `paths` list of absolute image paths or ephemeral image refs, `query` optional string describing what to extract, `raw` optional boolean to force direct image injection
6 +Input schema for tool_args:
7 +```json
8 +{
9 + "type": "object",
10 + "properties": {
11 + "paths": {
12 + "type": "array",
13 + "items": {"type": "string"},
14 + "description": "Absolute image paths or ephemeral image refs. A single bare string is also accepted."
15 + },
16 + "query": {
17 + "type": "string",
18 + "description": "Focused instruction for the vision model, e.g. 'read the top-right error toast' or 'locate the login button and give its position'. If omitted, a generic precise description is returned."
19 + },
20 + "raw": {
21 + "type": "boolean",
22 + "description": "If true, bypass delegation and inject the images directly into the main model (only when the main model can see images)."
23 + }
24 + },
25 + "required": ["paths"],
26 + "additionalProperties": false
27 +}
28 +```
29 +rules:
30 +- the focus argument is named exactly `query` — never `Prompt`, `question`, or any other name
31 +- `paths` as a JSON array even for one image: `{"paths": ["/path/to/image.png"]}` — a bare string is also accepted
32 +- a dedicated Vision Model IS configured: you will NOT see the images; vision_load returns a concise text capsule answering your `query`
33 +- write a focused `query` (e.g. "read the top-right error toast", "locate the login button and give its position"); if omitted, a generic precise description is returned
34 +- `raw=true` (default false) bypasses delegation and injects images directly — only use when Main can see images and truly needs pixels (e.g. side-by-side comparison)
35 +- load all relevant images in one call when comparing screenshots or pages; only bitmaps are supported
36 +- large images are auto-compressed before sending
37 +example:
38 +```json
39 +{
40 + "thoughts": [
41 + "I need to inspect the screenshot before answering."
42 + ],
43 + "headline": "Loading screenshot for visual analysis",
44 + "tool_name": "vision_load",
45 + "tool_args": {
46 + "paths": ["/path/to/screenshot.png"],
47 + "query": "read any error message visible in the top right"
48 + }
49 +}
50 +```
plugins/_vision_sidecar/scripts/enable_vision_slot.py new
+431
@@ -0,0 +1,431 @@
1 +#!/usr/bin/env python3
2 +"""Vision Sidecar - add the Vision Model slot to Model Presets (one-time, optional).
3 +
4 +Self-contained: no git, no patch(1) needed. Works from any directory.
5 +Usage:
6 + python3 enable_vision_slot.py apply (idempotent)
7 + python3 enable_vision_slot.py --status show state only
8 + python3 enable_vision_slot.py --restore restore backups (.bak)
9 +Docker: run INSIDE the Agent Zero container:
10 + docker exec -it <container> python3 usr/plugins/vision_sidecar/scripts/enable_vision_slot.py
11 +Override root: A0_ROOT=/path/to/agent-zero python3 enable_vision_slot.py
12 +"""
13 +from __future__ import annotations
14 +import os, shutil, sys
15 +from pathlib import Path
16 +
17 +REPO = "https://github.com/GreifMax/a0-vision-sidecar"
18 +
19 +def find_a0_root(script_path: Path) -> Path | None:
20 + env = os.environ.get("A0_ROOT", "").strip()
21 + cands: list[Path] = []
22 + # Priority: A0_ROOT env, then CWD walk-up (explicit user intent), then script walk-up
23 + if env:
24 + cands.append(Path(env))
25 + cur = Path.cwd().resolve()
26 + for _ in range(8):
27 + cands.append(cur)
28 + cur = cur.parent
29 + cur = script_path.resolve()
30 + for _ in range(8):
31 + cands.append(cur)
32 + cur = cur.parent
33 + for c in cands:
34 + try:
35 + c = c.resolve()
36 + except Exception:
37 + continue
38 + if (c / "plugins" / "_model_config" / "helpers" / "model_config.py").is_file():
39 + return c
40 + return None
41 +
42 +# --- edits ---------------------------------------------------------------
43 +V = "Optional dedicated model for vision_load - used when Main has no vision. Leave empty to use Main's vision."
44 +
45 +PY_EDITS = [
46 + dict(op="insert_after", anchor=' "chat": "chat_model",',
47 + payload=' "vision": "vision_model",\n', done='"vision": "vision_model"'),
48 + dict(op="insert_after", anchor="IMPLICIT_PRESET_SLOT_DEFAULTS = {",
49 + payload=' "vision": {\n "rl_requests": 0,\n "rl_input": 0,\n "rl_output": 0,\n "kwargs": {},\n },\n',
50 + done='"vision": {\n "rl_requests": 0'),
51 + dict(op="insert_after", anchor=" slot_clean = _strip_ui_fields(slot_config, strip_api_key=True)",
52 + payload=' if slot == "vision" and _slot_has_identity(slot_clean):\n slot_clean["vision"] = True\n if "max_embeds" not in slot_clean:\n slot_clean["max_embeds"] = 10\n',
53 + done='slot_clean["vision"] = True'),
54 + dict(op="replace", anchor='for section_name in ("chat_model", "utility_model", "embedding_model"):'.replace('"','\"'),
55 + new='for section_name in ("chat_model", "vision_model", "utility_model", "embedding_model"):'.replace('"','\"'),
56 + done='"chat_model", "vision_model", "utility_model"'),
57 + dict(op="insert_before", anchor="def is_chat_override_allowed",
58 + payload='def get_vision_model_config(agent=None) -> dict:\n """Vision model config from the vision preset slot (Vision Sidecar)."""\n return get_effective_config(agent).get("vision_model", {})\n\n\n',
59 + done="def get_vision_model_config"),
60 + dict(op="replace",
61 + anchor=""" slot_config = _get_preset_slot_config(preset, slot)
62 + if not _should_apply_preset_slot(slot, slot_config):
63 + continue
64 + config[section] = _merge_model_slot(
65 + slot,
66 + config.get(section, {}),
67 + slot_config,
68 + strip_api_key=strip_api_key,
69 + )""",
70 + new=""" slot_config = _get_preset_slot_config(preset, slot)
71 + if not _should_apply_preset_slot(slot, slot_config):
72 + if slot == "vision":
73 + # Vision Sidecar: vision is strictly per-preset — never inherit
74 + # the Default preset's vision model into other presets.
75 + config[section] = {}
76 + continue
77 + base_slot = config.get(section, {})
78 + if slot == "vision":
79 + # Vision Sidecar: merge vision over an empty base so a preset's own
80 + # vision model replaces, never accumulates on top of, another one.
81 + base_slot = {}
82 + config[section] = _merge_model_slot(
83 + slot,
84 + base_slot,
85 + slot_config,
86 + strip_api_key=strip_api_key,
87 + )""",
88 + done='if slot == "vision":'),
89 +]
90 +
91 +STORE_EDITS = [
92 + dict(op="insert_after", anchor=" { key: 'chat_model', title: 'Main Model', desc: 'Primary model for chat, reasoning, and browser tasks.' },",
93 + payload=" { key: 'vision_model', title: 'Vision Model', desc: \"" + V + "'s vision.' },\n".replace("'s vision.' }", "\\'s vision.' }", 0) if False else " { key: 'vision_model', title: 'Vision Model', desc: \"Optional dedicated model for vision_load - used when Main has no vision. Leave empty to use Main\\'s vision.\" },\n",
94 + done="key: 'vision_model'"),
95 + dict(op="insert_after", anchor="const IMPLICIT_PRESET_SLOT_DEFAULTS = {",
96 + payload=" vision: {\n rl_requests: 0,\n rl_input: 0,\n rl_output: 0,\n kwargs: {},\n },\n",
97 + done=" vision: {"),
98 + dict(op="insert_after", anchor=" ['chat', 'chat_model'],",
99 + payload=" ['vision', 'vision_model'],\n", done="['vision', 'vision_model']"),
100 + dict(op="insert_after", anchor=" chat_model: slot(rawDefault.chat),",
101 + payload=" vision_model: slot(rawDefault.vision),\n", done="vision_model: slot(rawDefault.vision)"),
102 + dict(op="insert_after", anchor=" chat: { ...slot(effective.chat_model), _kwargs_text: kwargsToText(effective.chat_model?.kwargs) },",
103 + payload=""" vision: {
104 + ...slot(effective.vision_model),
105 + // Vision Sidecar: display defaults (64000 / 0.7) when unset
106 + ...(hasModelIdentity(effective.vision_model || {})
107 + ? { ctx_length: Number(effective.vision_model?.ctx_length) || 64000, ctx_history: Number(effective.vision_model?.ctx_history ?? 0.7) }
108 + : { ctx_length: 64000, ctx_history: 0.7 }),
109 + _kwargs_text: kwargsToText(effective.vision_model?.kwargs),
110 + },
111 +""",
112 + done="display defaults (64000 / 0.7) when unset"),
113 + dict(op="replace",
114 + anchor=""" const slot = preset?.[slotKey];
115 + if (!slot || typeof slot !== 'object') continue;
116 + if (!hasModelIdentity(slot)) continue;
117 + config[sectionKey] = mergeModelSlot(config[sectionKey] || {}, slot, stripApiKey, slotKey);""",
118 + new=""" const slot = preset?.[slotKey];
119 + const isVision = slotKey === 'vision';
120 + if (!slot || typeof slot !== 'object') {
121 + if (isVision) config.vision_model = {}; // Vision Sidecar: never inherit from Default
122 + continue;
123 + }
124 + if (!hasModelIdentity(slot)) {
125 + if (isVision) config.vision_model = {}; // Vision Sidecar: never inherit from Default
126 + continue;
127 + }
128 + // Vision Sidecar: merge vision over an empty base (strictly per-preset)
129 + config[sectionKey] = mergeModelSlot(isVision ? {} : (config[sectionKey] || {}), slot, stripApiKey, slotKey);""",
130 + done="const isVision = slotKey === 'vision';"),
131 + dict(op="replace", anchor=" chat: { provider: '', name: '', api_base: '', kwargs: {}, _kwargs_text: '' },",
132 + new=" chat: { provider: '', name: '', api_base: '', ctx_length: 200000, ctx_history: 0.7, kwargs: {}, _kwargs_text: '' },\n vision: { provider: '', name: '', api_base: '', ctx_length: 64000, ctx_history: 0.7, vision: true, max_embeds: 10, kwargs: {}, _kwargs_text: '' },",
133 + done="vision: { provider: '', name: '', api_base: '', ctx_length: 64000"),
134 + dict(op="replace", anchor=" utility: { provider: '', name: '', api_base: '', kwargs: {}, _kwargs_text: '' },",
135 + new=" utility: { provider: '', name: '', api_base: '', ctx_length: 128000, ctx_input: 0.7, kwargs: {}, _kwargs_text: '' },",
136 + done="utility: { provider: '', name: '', api_base: '', ctx_length: 128000"),
137 + dict(op="insert_before", anchor=" const preset = {",
138 + payload=" // ensure Vision slot defaults for new presets (64000 / 0.7)\n if (!base.vision || typeof base.vision.ctx_length === 'undefined') {\n base.vision = { provider: '', name: '', api_base: '', ctx_length: 64000, ctx_history: 0.7, vision: true, max_embeds: 10, kwargs: {}, _kwargs_text: '', ...(base.vision || {}) };\n }\n",
139 + done="base.vision = { provider: ''"),
140 + dict(op="replace", anchor=" for (const slot of ['chat', 'utility']) {",
141 + new=" for (const slot of ['chat', 'vision', 'utility']) {",
142 + done="['chat', 'vision', 'utility']"),
143 + dict(op="replace", anchor=" if (hasModelIdentity(rest)) c[slot] = rest;",
144 + new=" if (hasModelIdentity(rest)) {\n if (slot === 'vision') { rest.vision = true; if (!('max_embeds' in rest)) rest.max_embeds = 10; }\n c[slot] = rest;\n }",
145 + done="rest.vision = true"),
146 + dict(op="insert_after", anchor=" { icon: 'chat', title: 'Main', cfg: preset?.chat, pList: chatP },",
147 + payload=" { icon: 'eye', title: 'Vision', cfg: preset?.vision, pList: chatP },\n",
148 + done="title: 'Vision'"),
149 + dict(op="replace",
150 + anchor=""" ].map(s => ({ icon: s.icon, title: s.title, provider: label(s.pList, s.cfg?.provider), name: s.cfg?.name || '\\u2014' }));""",
151 + new=""" ].map(s => {
152 + const overridden = s.title === 'Vision'
153 + && (s.cfg?.provider || s.cfg?.name)
154 + && preset?.chat?.vision
155 + && preset?.chat?.vision_override;
156 + return {
157 + icon: s.icon,
158 + title: s.title,
159 + provider: label(s.pList, s.cfg?.provider),
160 + name: s.cfg?.name || '\\u2014',
161 + note: overridden ? 'Overwritten by Main' : '',
162 + };
163 + });""",
164 + done="note: overridden"),
165 +]
166 +
167 +OVERVIEW_EDITS = [
168 + dict(op="replace",
169 + anchor=""" <span class="model-preset-identity">
170 + <span class="model-preset-provider" x-text="model.provider"></span>
171 + <span class="model-preset-separator">/</span>
172 + <span x-text="model.name"></span>
173 + </span>""",
174 + new=""" <span class="model-preset-identity">
175 + <template x-if="model.note">
176 + <span class="model-preset-note" x-text="model.note"></span>
177 + </template>
178 + <template x-if="!model.note">
179 + <span class="model-preset-plain">
180 + <span class="model-preset-provider" x-text="model.provider"></span>
181 + <span class="model-preset-separator">/</span>
182 + <span x-text="model.name"></span>
183 + </span>
184 + </template>
185 + </span>""",
186 + done="model-preset-note"),
187 + dict(op="replace",
188 + anchor=""" .model-preset-separator {
189 + margin: 0 0.25rem;
190 + }""",
191 + new=""" .model-preset-separator {
192 + margin: 0 0.25rem;
193 + }
194 +
195 + .model-preset-note {
196 + opacity: 0.65;
197 + font-style: italic;
198 + }""",
199 + done=".model-preset-note {"),
200 +]
201 +
202 +MAIN_SECTION = (
203 +" <section class=\"preset-model-section\">\n"
204 +" <div class=\"preset-model-heading\">\n"
205 +" <div class=\"section-title\">Vision Model</div>\n"
206 +" <div class=\"section-description\">" + V + "</div>\n"
207 +" </div>\n"
208 +" <div x-data=\"{ get model() { return selectedPreset.vision; }, modelType: 'vision', providers: $store.modelConfig.chatProviders, searchType: 'chat', apiKeyMode: 'store', get providerFallback() { return selectedPreset.chat.provider; }, get apiBaseFallback() { return selectedPreset.chat.api_base; } }\">\n"
209 +" <x-component path=\"/plugins/_model_config/webui/model-field.html\"></x-component>\n"
210 +" </div>\n"
211 +" </section>\n"
212 +"\n"
213 +)
214 +
215 +MAIN_EDITS = [
216 + dict(op="insert_before_back", anchor="Utility Model</div>", back='<section class="preset-model-section">',
217 + payload=MAIN_SECTION, done="section-title\">Vision Model</div>"),
218 +]
219 +
220 +SWITCHER_BLOCK = (
221 +" <template x-if=\"preset.vision?.name && !(preset.chat?.vision && preset.chat?.vision_override)\">\n"
222 +" <div class=\"model-switcher-model-row\">\n"
223 +" <span class=\"model-switcher-model-label\">Vision</span>\n"
224 +" <span class=\"model-switcher-model-value\">\n"
225 +" <span x-text=\"preset.vision.provider\" style=\"opacity:0.5;\"></span>\n"
226 +" <span style=\"opacity:0.3; margin:0 3px;\">/</span>\n"
227 +" <span x-text=\"preset.vision.name\"></span>\n"
228 +" </span>\n"
229 +" </div>\n"
230 +" </template>\n"
231 +)
232 +SWITCHER_EDITS = [
233 + dict(op="replace_optional", anchor='<template x-if="preset.vision?.name">',
234 + new='<template x-if="preset.vision?.name && !(preset.chat?.vision && preset.chat?.vision_override)">',
235 + done="preset.vision?.name && !(preset.chat?.vision && preset.chat?.vision_override)"),
236 + dict(op="insert_before", anchor='<template x-if="preset.utility?.name">', payload=SWITCHER_BLOCK,
237 + done="preset.vision?.name && !(preset.chat?.vision && preset.chat?.vision_override)"),
238 +]
239 +
240 +FIELD_EDITS = [
241 + dict(op="replace_occurrence", anchor='<template x-if="modelType === \'chat\'">', occurrence=2,
242 + new='<template x-if="modelType === \'chat\' || modelType === \'vision\'">',
243 + done="modelType === 'chat' || modelType === 'vision'"),
244 + dict(op="replace", anchor='<template x-if="model.vision">',
245 + new='<template x-if="model.vision || modelType === \'vision\'">',
246 + done="model.vision || modelType === 'vision'"),
247 + dict(op="replace", anchor="Maximum number of embedded images used by the chat model. Set to 0 for unlimited.",
248 + new="Maximum number of embedded images used by the model. Set to 0 for unlimited.",
249 + done="used by the model. Set to 0"),
250 +]
251 +
252 +FIELD_OVERRIDE_BLOCK = (""" <!-- Vision Model override (Main chat only) -->
253 + <template x-if="modelType === 'chat' && model.vision">
254 + <div class="field">
255 + <div class="field-label">
256 + <div class="field-title">Overrides Vision Model</div>
257 + <div class="field-description">If enabled, this model's native vision is always used and the preset's Vision Model is ignored. If disabled, the dedicated Vision Model handles vision when configured.</div>
258 + </div>
259 + <div class="field-control">
260 + <label class="toggle">
261 + <input type="checkbox" x-model="model.vision_override" />
262 + <span class="toggler"></span>
263 + </label>
264 + </div>
265 + </div>
266 + </template>
267 +
268 +""")
269 +
270 +FIELD_EDITS.append(dict(op="insert_before", anchor="<!-- Context window size (main and utility only) -->",
271 + payload=FIELD_OVERRIDE_BLOCK, done="Overrides Vision Model"))
272 +
273 +RESPONSES_EDITS = [
274 + # New A0 (2026-08+): stock _vision_tool_prompt exists -> make it Vision Sidecar aware.
275 + dict(op="replace_optional",
276 + anchor='def _vision_tool_prompt(agent: Any) -> str:\n try:\n from plugins._model_config.helpers.model_config import get_chat_model_config\n\n if not get_chat_model_config(agent).get("vision", False):\n return ""\n return agent.read_prompt("agent.system.tools_vision.md")\n except Exception:\n return ""\n\n\n',
277 + new='def _vision_tool_prompt(agent: Any) -> str:\n try:\n from plugins._model_config.helpers.model_config import get_chat_model_config\n except Exception:\n return ""\n # Vision Sidecar: a dedicated Vision Model takes precedence over main vision.\n try:\n from usr.plugins.vision_sidecar.helpers.vision_model import has_vision_model\n\n if has_vision_model(agent):\n return agent.read_prompt("vision_sidecar.delegated.md")\n except Exception:\n pass\n try:\n if not get_chat_model_config(agent).get("vision", False):\n return ""\n return agent.read_prompt("agent.system.tools_vision.md")\n except Exception:\n return ""\n\n\n',
278 + done='Vision Sidecar: a dedicated Vision Model takes precedence over main vision.'),
279 + # Older A0: no _vision_tool_prompt -> insert the sidecar-aware function.
280 + dict(op="insert_before", anchor="def _include_local_tool_prompt(",
281 + payload='def _vision_tool_prompt(agent: Any) -> str:\n try:\n from plugins._model_config.helpers.model_config import get_chat_model_config\n except Exception:\n return ""\n # Vision Sidecar: a dedicated Vision Model takes precedence over main vision.\n try:\n from usr.plugins.vision_sidecar.helpers.vision_model import has_vision_model\n\n if has_vision_model(agent):\n return agent.read_prompt("vision_sidecar.delegated.md")\n except Exception:\n pass\n try:\n if not get_chat_model_config(agent).get("vision", False):\n return ""\n return agent.read_prompt("agent.system.tools_vision.md")\n except Exception:\n return ""\n\n\n',
282 + done='Vision Sidecar: a dedicated Vision Model takes precedence over main vision.'),
283 + # Older A0: hook the vision prompt into _local_tool_prompts.
284 + dict(op="replace_optional",
285 + anchor=' tool_name = _tool_name_from_prompt(prompt, fallback=fallback_name)\n if not _include_local_tool_prompt(agent, tool_name):\n continue\n result.append((tool_name, prompt))\n return result',
286 + new=' tool_name = _tool_name_from_prompt(prompt, fallback=fallback_name)\n if not _include_local_tool_prompt(agent, tool_name):\n continue\n result.append((tool_name, prompt))\n\n vision_prompt = _vision_tool_prompt(agent)\n if vision_prompt:\n result.append(("vision_load", vision_prompt))\n return result',
287 + done='vision_prompt = _vision_tool_prompt(agent)'),
288 +]
289 +
290 +PARALLEL_EDITS = [
291 + dict(op="replace",
292 + anchor=' try:\n job.log_item.update(content=content)\n except Exception:\n pass\n',
293 + new=' try:\n # Avoid duplicating text that the tool already placed in the "result"\n # kvps row of the step table. When the tool\'s result row matches the\n # job result, skip the body content write so the step shows the text\n # only once (inside the Result row).\n existing_result = (job.log_item.kvps or {}).get("result") if job.state == "success" else None\n if existing_result is not None and str(existing_result).strip() == str(content).strip():\n return\n job.log_item.update(content=content)\n except Exception:\n pass\n',
294 + done="Avoid duplicating text that the tool already placed"),
295 +]
296 +
297 +TOOLSPROMPT_EDITS = [
298 + # Vision block is identical in old and new stock.
299 + dict(op="replace",
300 + anchor=' # vision support\n from plugins._model_config.helpers.model_config import get_chat_model_config\n\n chat_cfg = get_chat_model_config(agent)\n if chat_cfg.get("vision", False):\n prompt += "\\n\\n" + agent.read_prompt("agent.system.tools_vision.md")\n',
301 + new=' # vision support (Vision Sidecar: dedicated Vision Model takes precedence)\n from plugins._model_config.helpers.model_config import get_chat_model_config\n\n chat_cfg = get_chat_model_config(agent)\n try:\n from usr.plugins.vision_sidecar.helpers.vision_model import has_vision_model\n\n has_sidecar = has_vision_model(agent)\n except Exception:\n has_sidecar = False\n if has_sidecar:\n prompt += "\\n\\n" + agent.read_prompt("vision_sidecar.delegated.md")\n elif chat_cfg.get("vision", False):\n prompt += "\\n\\n" + agent.read_prompt("agent.system.tools_vision.md")\n',
302 + done='(Vision Sidecar: dedicated Vision Model takes precedence)'),
303 +]
304 +
305 +FILES = [
306 + ("plugins/_model_config/helpers/model_config.py", PY_EDITS),
307 + ("plugins/_model_config/webui/model-config-store.js", STORE_EDITS),
308 + ("plugins/_model_config/webui/main.html", MAIN_EDITS),
309 + ("plugins/_model_config/extensions/webui/chat-input-progress-start/model-switcher.html", SWITCHER_EDITS),
310 + ("plugins/_model_config/webui/model-field.html", FIELD_EDITS),
311 + ("plugins/_model_config/webui/preset-overview.html", OVERVIEW_EDITS),
312 + ("helpers/responses_tools.py", RESPONSES_EDITS),
313 + ("extensions/python/system_prompt/_11_tools_prompt.py", TOOLSPROMPT_EDITS),
314 + ("helpers/parallel_tools.py", PARALLEL_EDITS),
315 +]
316 +
317 +def line_start(text: str, idx: int) -> int:
318 + return text.rfind("\n", 0, idx) + 1
319 +
320 +def apply_edit(text: str, e: dict):
321 + done = e.get("done", "")
322 + if done and done in text:
323 + return text, "already"
324 + op, anchor = e["op"], e["anchor"]
325 + if op == "replace":
326 + if anchor not in text: return text, "ANCHOR NOT FOUND"
327 + return text.replace(anchor, e["new"], 1), "ok"
328 + if op == "replace_occurrence":
329 + occ, cur, seen = e["occurrence"], -1, 0
330 + while True:
331 + cur = text.find(anchor, cur + 1)
332 + if cur < 0: break
333 + seen += 1
334 + if seen == occ:
335 + return text[:cur] + e["new"] + text[cur + len(anchor):], "ok"
336 + return text, f"OCCURRENCE {occ} NOT FOUND ({seen} total)"
337 + if op == "replace_optional":
338 + if anchor not in text:
339 + return text, "skip"
340 + return text.replace(anchor, e["new"], 1), "ok"
341 + if op == "insert_after":
342 + idx = text.find(anchor)
343 + if idx < 0: return text, "ANCHOR NOT FOUND"
344 + nl = text.find("\n", idx + len(anchor))
345 + ins = nl + 1 if nl >= 0 else len(text)
346 + return text[:ins] + e["payload"] + text[ins:], "ok"
347 + if op == "insert_before":
348 + idx = text.find(anchor)
349 + if idx < 0: return text, "ANCHOR NOT FOUND"
350 + ins = line_start(text, idx)
351 + return text[:ins] + e["payload"] + text[ins:], "ok"
352 + if op == "insert_before_back":
353 + pos = text.find(e["anchor"])
354 + if pos < 0: return text, "ANCHOR NOT FOUND"
355 + back = text.rfind(e["back"], 0, pos)
356 + if back < 0: return text, "BACK ANCHOR NOT FOUND"
357 + ins = line_start(text, back)
358 + return text[:ins] + e["payload"] + text[ins:], "ok"
359 + return text, "UNKNOWN OP"
360 +
361 +def run(status_only: bool = False, restore: bool = False) -> int:
362 + root = find_a0_root(Path(__file__))
363 + print(f"Vision Sidecar patcher | repo: {REPO}")
364 + if root is None:
365 + print("ERROR: Agent Zero root not found (plugins/_model_config/helpers/model_config.py).", file=sys.stderr)
366 + print(" - run from inside the Agent Zero folder (or its container):", file=sys.stderr)
367 + print(" docker exec -it <container> python3 usr/plugins/vision_sidecar/scripts/enable_vision_slot.py", file=sys.stderr)
368 + print(" - or set A0_ROOT=/path/to/agent-zero", file=sys.stderr)
369 + return 1
370 + print(f"A0 root: {root}\n")
371 + any_fail, changed = False, []
372 + for rel, edits in FILES:
373 + p = root / rel
374 + if not p.is_file():
375 + print(f" MISSING FILE: {rel}"); any_fail = True; continue
376 + bak = p.with_suffix(p.suffix + ".vision_sidecar.bak")
377 + if restore:
378 + if bak.is_file():
379 + current = ""
380 + try:
381 + current = p.read_text(encoding="utf-8")
382 + except Exception:
383 + pass
384 + still_patched = any(e.get("done", "") and e["done"] in current for e in edits)
385 + if still_patched:
386 + shutil.copy2(bak, p); bak.unlink(); print(f" restored {rel}")
387 + else:
388 + print(f" skipped {rel} (no Vision Sidecar edits present — core may have been updated; keeping current file)")
389 + else:
390 + print(f" no backup {rel}")
391 + continue
392 + text = p.read_text(encoding="utf-8")
393 + report, modified = [], False
394 + for e in edits:
395 + text, res = apply_edit(text, e)
396 + if res == "ok": modified = True
397 + report.append(res)
398 + if status_only:
399 + print(f" {rel}: " + ", ".join(report)); continue
400 + if modified:
401 + if not bak.is_file():
402 + shutil.copy2(p, bak)
403 + p.write_text(text, encoding="utf-8")
404 + changed.append(rel)
405 + bad = [r for r in report if r not in ("ok", "already", "skip")]
406 + icon = "FAIL" if bad else ("PATCHED" if modified else "already")
407 + print(f" {icon:8} {rel}" + (f" ({'; '.join(bad)})" if bad else ""))
408 + if bad: any_fail = True
409 + if restore: return 0
410 + print()
411 + if any_fail:
412 + print("Some anchors were not found - your A0 version may be newer/older than supported.")
413 + print(f"Restore originals with --restore, or open an issue: {REPO}/issues")
414 + return 1
415 + if changed:
416 + print("Done. Restart Agent Zero, then hard-refresh the browser (Ctrl+Shift+R).")
417 + print("Model Presets now shows: Main / Vision / Utility / Embedding.")
418 + return 0
419 + print("Everything already patched - nothing to do.")
420 + return 0
421 +
422 +def main():
423 + status_only = "--status" in sys.argv
424 + restore = "--restore" in sys.argv
425 + rc = run(status_only=status_only, restore=restore)
426 + if rc:
427 + sys.exit(rc)
428 +
429 +
430 +if __name__ == "__main__":
431 + main()
plugins/_vision_sidecar/scripts/enable_vision_slot.sh new
+25
@@ -0,0 +1,25 @@
1 +#!/usr/bin/env bash
2 +# Vision Sidecar - add the Vision Model slot to Model Presets (one-time, optional)
3 +# Thin wrapper: all logic lives in enable_vision_slot.py (no git / patch(1) needed).
4 +# Run AFTER installing the plugin, from ANY directory:
5 +# bash usr/plugins/vision_sidecar/scripts/enable_vision_slot.sh
6 +# bash usr/plugins/vision_sidecar/scripts/enable_vision_slot.sh --status
7 +# bash usr/plugins/vision_sidecar/scripts/enable_vision_slot.sh --restore
8 +# Docker (run INSIDE the container):
9 +# docker exec -it <container> bash usr/plugins/vision_sidecar/scripts/enable_vision_slot.sh
10 +# Force a root when auto-detect fails: A0_ROOT=/path/to/agent-zero bash ./enable_vision_slot.sh
11 +set -u
12 +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
13 +PY="$SCRIPT_DIR/enable_vision_slot.py"
14 +if [ ! -f "$PY" ]; then
15 + echo "ERROR: enable_vision_slot.py not found next to this script." >&2
16 + exit 1
17 +fi
18 +if command -v python3 >/dev/null 2>&1; then
19 + exec python3 "$PY" "$@"
20 +elif command -v python >/dev/null 2>&1; then
21 + exec python "$PY" "$@"
22 +else
23 + echo "ERROR: python3 not found in PATH." >&2
24 + exit 1
25 +fi
plugins/_vision_sidecar/thumbnail.jpg
Binary files /dev/null and b/plugins/_vision_sidecar/thumbnail.jpg differ
plugins/_vision_sidecar/tools/vision_load.py new
+422
@@ -0,0 +1,422 @@
1 +from helpers.print_style import PrintStyle
2 +from helpers.tool import Tool, Response
3 +from helpers import runtime, files, plugins, ephemeral_images, images, chat_media
4 +from mimetypes import guess_type
5 +from helpers import history
6 +
7 +TOKENS_ESTIMATE = 1500
8 +
9 +
10 +class VisionLoad(Tool):
11 + async def execute(self, paths: list[str] | str = [], query: str = "", raw: bool = False, **kwargs) -> Response:
12 + # State for after_execution
13 + self.images_dict = {}
14 + self.loaded_paths: list[str] = []
15 + self.skipped_paths: list[str] = []
16 + self._arg_error: str | None = None
17 + self._is_delegated: bool = False
18 + self._delegated_capsule: str | None = None
19 + self._delegation_error: str | None = None
20 + self.query = str(query or "").strip()
21 + self._raw = bool(raw) if isinstance(raw, bool) else str(raw).lower() in ("true", "1", "yes")
22 +
23 + # --- Phase 0: normalize prompt-style aliases into `query` ---
24 + # Some models send a differently-named focus argument ("Prompt", "question",
25 + # "instruction", ...) despite the schema. Map any alias onto `query` so the
26 + # vision model always receives the intended focus text.
27 + _alias_map = {
28 + "prompt": "query",
29 + "prompt_text": "query",
30 + "question": "query",
31 + "instruction": "query",
32 + "instructions": "query",
33 + "focus": "query",
34 + "request": "query",
35 + "force_raw": "raw",
36 + }
37 + _canonical: dict = {}
38 + for _k in list(kwargs.keys()):
39 + _target = _alias_map.get(str(_k).strip().lower())
40 + if _target:
41 + _canonical.setdefault(_target, kwargs.pop(_k))
42 + if not self.query:
43 + _alias_q = _canonical.get("query")
44 + if isinstance(_alias_q, str) and _alias_q.strip():
45 + self.query = _alias_q.strip()
46 + if not self._raw:
47 + _alias_r = _canonical.get("raw")
48 + if isinstance(_alias_r, bool):
49 + self._raw = _alias_r
50 + elif isinstance(_alias_r, str) and _alias_r.strip().lower() in ("true", "1", "yes"):
51 + self._raw = True
52 +
53 + # --- Phase 1: tolerant coerce paths ---
54 + coerce = paths
55 + if isinstance(coerce, str):
56 + s = coerce.strip()
57 + if not s:
58 + coerce = []
59 + elif s.startswith("["):
60 + # handle LLM/harness mistakenly sending JSON-encoded array as string
61 + try:
62 + import json as _json
63 + parsed = _json.loads(s)
64 + if isinstance(parsed, list):
65 + coerce = parsed
66 + else:
67 + coerce = [s]
68 + except Exception:
69 + # fallback: try single-quoted / python literal
70 + try:
71 + import ast as _ast
72 + parsed = _ast.literal_eval(s)
73 + if isinstance(parsed, list):
74 + coerce = parsed
75 + else:
76 + coerce = [s]
77 + except Exception:
78 + coerce = [s]
79 + elif (s.startswith('"') and s.endswith('"')) or (s.startswith("'") and s.endswith("'")):
80 + # harness double-quoted a single path: "...png" -> strip outer quotes
81 + coerce = [s[1:-1].strip()]
82 + else:
83 + coerce = [s]
84 + elif isinstance(coerce, (list, tuple)):
85 + # keep as is, will stringify elements later
86 + coerce = list(coerce)
87 + elif coerce is None:
88 + coerce = []
89 + else:
90 + self._arg_error = (
91 + f"vision_load error: `paths` must be a list of strings, e.g. {{\"paths\": [\"/a.png\"]}} — got {type(paths).__name__}. "
92 + f"Use an array even for one image."
93 + )
94 + return Response(message="dummy", break_loop=False)
95 +
96 + # Normalize elements to strings
97 + try:
98 + # strip per-element surrounding quotes (defensive)
99 + def _strip_outer_q(v: str) -> str:
100 + v = v.strip()
101 + if len(v) >= 2 and ((v[0]=='"' and v[-1]=='"') or (v[0]=="\'" and v[-1]=="\'")):
102 + return v[1:-1].strip()
103 + return v
104 + paths_list = [_strip_outer_q(str(p or "")) for p in coerce]
105 + except Exception as e:
106 + self._arg_error = f"vision_load error: invalid `paths` value: {e}"
107 + return Response(message="dummy", break_loop=False)
108 +
109 + max_embeds = self._get_max_embeds()
110 + requested = [
111 + (str(path or "").strip(), self._display_input_path(str(path or "").strip(), idx + 1))
112 + for idx, path in enumerate(paths_list)
113 + ]
114 + limited_paths = requested if max_embeds <= 0 else requested[-max_embeds:]
115 + self.skipped_paths = (
116 + [display for _, display in requested[:-max_embeds]]
117 + if max_embeds > 0 and len(requested) > max_embeds
118 + else []
119 + )
120 +
121 + for idx, (path, display_path) in enumerate(limited_paths):
122 + if not path:
123 + continue
124 + if ephemeral_images.is_ref(path):
125 + image = ephemeral_images.consume_image(path, context_id=self._context_id())
126 + if image is None:
127 + continue
128 + display = image.display_name or display_path
129 + stored_ref = self._store_ephemeral_image(image)
130 + if stored_ref:
131 + self.images_dict[display] = stored_ref
132 + self.loaded_paths.append(display)
133 + continue
134 + if self._is_data_image_url(path):
135 + stored_ref = self._store_data_url(path, preferred_name=f"vision-load-{idx + 1}.png")
136 + if stored_ref:
137 + self.images_dict[display_path] = stored_ref
138 + self.loaded_paths.append(display_path)
139 + continue
140 + if not await runtime.call_development_function(files.exists, str(path)):
141 + continue
142 + if path not in self.images_dict:
143 + mime_type, _ = guess_type(str(path))
144 + if mime_type and mime_type.startswith("image/"):
145 + try:
146 + stored_ref = self._store_local_image(path, preferred_name=files.basename(path))
147 + self.images_dict[display_path] = stored_ref
148 + self.loaded_paths.append(display_path)
149 + except (FileNotFoundError, OSError, ValueError):
150 + continue
151 +
152 + # --- Phase 2: delegated vision model call (if configured and not raw) ---
153 + # We defer the actual LLM call to after_execution to keep execute fast,
154 + # but we can also do it here. Doing it in after_execution preserves
155 + # the original history injection pattern. So just mark intent here.
156 + # The actual call happens in after_execution when we know images_dict.
157 + return Response(message="dummy", break_loop=False)
158 +
159 + def _get_max_embeds(self) -> int:
160 + # Prefer the effective (preset-aware) chat config; fall back to raw plugin config.
161 + try:
162 + from plugins._model_config.helpers.model_config import get_chat_model_config
163 + chat_cfg = get_chat_model_config(self.agent) or {}
164 + except Exception:
165 + cfg = plugins.get_plugin_config("_model_config", agent=self.agent) or {}
166 + chat_cfg = cfg.get("chat_model", {}) or {}
167 + try:
168 + return int(chat_cfg.get("max_embeds", 10) or 0)
169 + except Exception:
170 + return 10
171 +
172 + def _main_has_vision(self) -> bool:
173 + # Whether the *main* chat model accepts image content. Injected image_url
174 + # blocks must never reach a text-only provider (400 "content.type invalid").
175 + try:
176 + from plugins._model_config.helpers.model_config import get_chat_model_config
177 + return bool(get_chat_model_config(self.agent).get("vision", False))
178 + except Exception:
179 + try:
180 + cfg = plugins.get_plugin_config("_model_config", agent=self.agent) or {}
181 + return bool((cfg.get("chat_model", {}) or {}).get("vision", False))
182 + except Exception:
183 + return False
184 +
185 + def _update_log(self, message: str) -> None:
186 + # Write the "Result" row (kvps) of the tool's log item — including inside
187 + # `parallel` workers. The parent's _update_parallel_child_log only writes
188 + # the plain body text (`content=`) of the same item; it never sets the
189 + # `result` row, so skipping this write would lose the Result row entirely.
190 + try:
191 + self.log.update(result=message)
192 + except Exception:
193 + pass
194 +
195 + def _context_id(self) -> str:
196 + return str(getattr(getattr(self.agent, "context", None), "id", "") or "").strip()
197 +
198 + def _store_ephemeral_image(self, image: ephemeral_images.EphemeralImage) -> str:
199 + context_id = self._context_id()
200 + if not context_id:
201 + return image.data_url
202 + source = chat_media.infer_source(image.ref, image.display_name)
203 + category = chat_media.category_for_source(source)
204 + saved = chat_media.save_image_base64(
205 + context_id=context_id,
206 + data=image.data,
207 + mime_type=image.mime,
208 + category=category,
209 + source=source,
210 + preferred_name=image.display_name,
211 + )
212 + return saved.a0_path
213 +
214 + def _store_data_url(self, data_url: str, *, preferred_name: str = "") -> str:
215 + context_id = self._context_id()
216 + if not context_id:
217 + return data_url
218 + source = chat_media.infer_source(data_url, preferred_name)
219 + category = chat_media.category_for_source(source)
220 + saved = chat_media.save_image_data_url(
221 + context_id=context_id,
222 + data_url=data_url,
223 + category=category,
224 + source=source,
225 + preferred_name=preferred_name,
226 + )
227 + return saved.a0_path
228 +
229 + def _store_local_image(self, path: str, *, preferred_name: str = "") -> str:
230 + context_id = self._context_id()
231 + if not context_id:
232 + return images.to_data_url(path)
233 + return chat_media.materialize_image_ref(
234 + context_id=context_id,
235 + url=path,
236 + source=chat_media.infer_source(path, preferred_name),
237 + preferred_name=preferred_name,
238 + )
239 +
240 + @staticmethod
241 + def _is_data_image_url(value: str) -> bool:
242 + normalized = str(value or "").strip().lower()
243 + return normalized.startswith("data:image/") and ";base64," in normalized
244 +
245 + @classmethod
246 + def _display_input_path(cls, value: str, index: int) -> str:
247 + if ephemeral_images.is_ref(value):
248 + return ephemeral_images.display_ref(value)
249 + if cls._is_data_image_url(value):
250 + prefix = value.split(",", 1)[0]
251 + return f"{prefix},<ephemeral-image-{index}>"
252 + return value
253 +
254 + async def before_execution(self, **kwargs):
255 + # Vision Sidecar: the focus argument is named `query`. Some models send a
256 + # differently-named key ("Prompt", "question", ...) despite the schema — fold
257 + # any alias into `query` in the logged args so the user sees exactly one row,
258 + # and always show the Query row in delegated mode even when it was omitted.
259 + try:
260 + if not isinstance(self.args, dict):
261 + self.args = dict(self.args or {})
262 + aliases = ("prompt", "prompt_text", "question", "instruction", "instructions", "focus", "request")
263 + alias_value = ""
264 + for key in list(self.args.keys()):
265 + if str(key).strip().lower() in aliases:
266 + val = self.args.pop(key)
267 + if isinstance(val, str) and val.strip() and not alias_value:
268 + alias_value = val.strip()
269 + if not self.args.get("query"):
270 + from usr.plugins.vision_sidecar.helpers.vision_model import has_vision_model
271 + if has_vision_model(self.agent):
272 + self.args["query"] = alias_value
273 + elif alias_value and not str(self.args.get("query") or "").strip():
274 + self.args["query"] = alias_value
275 + except Exception:
276 + pass
277 + await super().before_execution(**kwargs)
278 +
279 + async def after_execution(self, response: Response, **kwargs):
280 + # Handle arg error first: return as tool_result error (not raise)
281 + if getattr(self, "_arg_error", None):
282 + msg = self._arg_error
283 + lid = getattr(getattr(self, 'log', None), 'id', '') or ''
284 + self.agent.hist_add_tool_result(self.name, msg, id=lid)
285 + response.message = msg # authoritative result: parallel workers return response.message
286 + PrintStyle(font_color="#1B4F72", background_color="white", padding=True, bold=True).print(
287 + f"{self.agent.agent_name}: Response from tool '{self.name}'"
288 + )
289 + PrintStyle(font_color="#E74C3C").print(msg)
290 + self._update_log(msg)
291 + return
292 +
293 + content = []
294 + loaded_count = len(self.loaded_paths)
295 + skipped_count = len(self.skipped_paths)
296 + loaded_summary = "\n".join(self.loaded_paths) if self.loaded_paths else "none"
297 + skipped_summary = "\n".join(self.skipped_paths) if self.skipped_paths else "none"
298 + summary = (
299 + f"Loaded images ({loaded_count}):\n{loaded_summary}\n\n"
300 + f"Skipped images ({skipped_count}, max {self._get_max_embeds()} loaded at a time according to model configuration):\n{skipped_summary}"
301 + )
302 +
303 + # Determine delegation. `raw=true` is honored only when the main model can
304 + # actually receive images; on a text-only main raw injection is impossible,
305 + # so we delegate to the Vision Model instead of leaking image blocks.
306 + main_has_vision = self._main_has_vision()
307 + should_delegate = False
308 + if self.images_dict and (not getattr(self, "_raw", False) or not main_has_vision):
309 + try:
310 + from usr.plugins.vision_sidecar.helpers.vision_model import has_vision_model, call_vision_model, get_behaviour
311 + if has_vision_model(self.agent):
312 + should_delegate = True
313 + except Exception:
314 + should_delegate = False
315 +
316 + if should_delegate:
317 + # Delegated path: query + images -> vision model -> text capsule
318 + # No RawMessage injection -> saves ~1500 tok/image on main context
319 + try:
320 + from usr.plugins.vision_sidecar.helpers.vision_model import call_vision_model, get_behaviour
321 + behaviour = get_behaviour(self.agent)
322 + # images_dict values are a0_path strings
323 + a0_paths = list(self.images_dict.values())
324 + capsule = await call_vision_model(
325 + self.agent,
326 + a0_paths,
327 + getattr(self, "query", ""),
328 + timeout=behaviour["timeout"],
329 + )
330 + self._delegated_capsule = (capsule or "").strip()
331 + self._is_delegated = True
332 + except Exception as e:
333 + self._delegation_error = str(e)[:2000]
334 + self._is_delegated = False
335 +
336 + if getattr(self, "_is_delegated", False) and self._delegated_capsule is not None:
337 + # Success delegated — stock-like one-liner: counts + Description
338 + flat = " ".join(self._delegated_capsule.split())
339 + message = (
340 + f"{loaded_count} images sent, {skipped_count} images skipped"
341 + f' - Description: "{flat}"'
342 + )
343 + lid = getattr(getattr(self, 'log', None), 'id', '') or ''
344 + self.agent.hist_add_tool_result(self.name, message, id=lid)
345 + response.message = message # authoritative result: parallel workers return response.message
346 + # Inject RawMessage thumbnails for UI parity with native vision when the
347 + # main preset declares vision support. The flag is the user's declaration
348 + # and governs: if it mislabels a text-only provider, the resulting provider
349 + # error is a configuration issue the user owns — the capsule still carries
350 + # the answer in the meantime. (Deliberate design decision, restored 0.7.7.)
351 + if self.images_dict and main_has_vision:
352 + content = []
353 + for path, image_path in self.images_dict.items():
354 + if image_path:
355 + content.append({"type": "image_url", "image_url": {"url": image_path}})
356 + else:
357 + content.append({"type": "text", "text": "Error processing image " + path})
358 + msg = history.RawMessage(raw_content=content, preview="<Image attachments loaded by path>")
359 + self.agent.hist_add_message(False, content=msg, tokens=TOKENS_ESTIMATE * len(content))
360 + PrintStyle(font_color="#1B4F72", background_color="white", padding=True, bold=True).print(
361 + f"{self.agent.agent_name}: Response from tool '{self.name}'"
362 + )
363 + PrintStyle(font_color="#85C1E9").print(message)
364 + self._update_log(message)
365 + return
366 +
367 + if getattr(self, "_delegation_error", None):
368 + # Delegation failed — surface error but do NOT inject images (main has no vision)
369 + combined = (
370 + summary
371 + + f"\n\n[Vision model error: {self._delegation_error}]\n"
372 + + "Tip: check Vision Sidecar Vision Model settings (provider/name/api_key) or retry with raw=true to inject images directly."
373 + )
374 + lid = getattr(getattr(self, 'log', None), 'id', '') or ''
375 + self.agent.hist_add_tool_result(self.name, combined, id=lid)
376 + message = f"Vision model error — {self._delegation_error[:200]}"
377 + response.message = message # authoritative result: parallel workers return response.message
378 + PrintStyle(font_color="#1B4F72", background_color="white", padding=True, bold=True).print(
379 + f"{self.agent.agent_name}: Response from tool '{self.name}'"
380 + )
381 + PrintStyle(font_color="#E74C3C").print(message)
382 + self._update_log(message)
383 + return
384 +
385 + # Legacy / raw / no vision_model path: inject images as RawMessage
386 + if self.images_dict and main_has_vision:
387 + lid = getattr(getattr(self, 'log', None), 'id', '') or ''
388 + self.agent.hist_add_tool_result(self.name, summary, id=lid)
389 + for path, image_path in self.images_dict.items():
390 + if image_path:
391 + content.append({"type": "image_url", "image_url": {"url": image_path}})
392 + else:
393 + content.append({"type": "text", "text": "Error processing image " + path})
394 + msg = history.RawMessage(raw_content=content, preview="<Image attachments loaded by path>")
395 + self.agent.hist_add_message(False, content=msg, tokens=TOKENS_ESTIMATE * len(content))
396 + elif self.images_dict:
397 + # No vision anywhere (no Vision Model, main has no vision): never inject
398 + # image blocks a text-only provider would reject.
399 + lid = getattr(getattr(self, 'log', None), 'id', '') or ''
400 + self.agent.hist_add_tool_result(
401 + self.name,
402 + summary
403 + + "\n\n[Images not injected: the main model has no vision and no Vision Model is configured for delegation.]",
404 + id=lid,
405 + )
406 + else:
407 + lid2 = getattr(getattr(self, 'log', None), 'id', '') or ''
408 + self.agent.hist_add_tool_result(
409 + self.name, summary if self.skipped_paths else "No images processed", id=lid2
410 + )
411 +
412 + message = (
413 + "No images processed"
414 + if not self.images_dict and not self.skipped_paths
415 + else f"{loaded_count} images loaded, {skipped_count} skipped"
416 + )
417 + response.message = message # authoritative result: parallel workers return response.message
418 + PrintStyle(font_color="#1B4F72", background_color="white", padding=True, bold=True).print(
419 + f"{self.agent.agent_name}: Response from tool '{self.name}'"
420 + )
421 + PrintStyle(font_color="#85C1E9").print(message)
422 + self._update_log(message)
plugins/_vision_sidecar/webui/config.html new
+55
@@ -0,0 +1,55 @@
1 +<html>
2 +<head>
3 + <title>Vision Sidecar Settings</title>
4 + <script type="module">
5 + import { store } from "/components/plugins/plugin-settings-store.js";
6 + </script>
7 +</head>
8 +<body>
9 + <div x-data>
10 + <div class="field">
11 + <div class="field-label">
12 + <div class="field-title">Vision Sidecar</div>
13 + <div class="field-description">Configure the optional dedicated <b>Vision Model</b> in <b>Settings → Model Presets → Vision Model</b>.<br/>Optional dedicated model for <code>vision_load</code> — used when Main has no vision. Leave empty to use Main's vision.</div>
14 + </div>
15 + </div>
16 + <div class="field" style="opacity:0.85; border:1px dashed var(--color-border); padding:0.8rem; border-radius:8px;">
17 + <div class="field-label">
18 + <div class="field-title" style="font-size:0.9rem;">Where to set Vision Model</div>
19 + <div class="field-description">Open <b>Settings → Model Presets</b> (Edit). Each preset now has <b>Main / Vision / Utility / Embedding</b>. Put your cheap vision helper (e.g. <code>gpt-4o-mini</code>, <code>qwen2-vl</code>) in the <b>Vision Model</b> section. Leave it empty to use Main's vision. Supports GLM 5.2/5.3, DeepSeek V4 Flash/Pro etc.</div>
20 + </div>
21 + </div>
22 + <hr style="margin:1rem 0; border:none; border-top:1px solid var(--color-border);" />
23 + <div class="field">
24 + <div class="field-label">
25 + <div class="field-title">Delegated system prompt</div>
26 + <div class="field-description">System message sent to the Vision Model alongside your query + images.</div>
27 + </div>
28 + <div class="field-control">
29 + <textarea x-model="config.behaviour.delegated_system" rows="3" placeholder="You are a precise vision analyst..."></textarea>
30 + </div>
31 + </div>
32 + <div class="field">
33 + <div class="field-label">
34 + <div class="field-title">Timeout (seconds)</div>
35 + <div class="field-description">How long to wait for the vision model.</div>
36 + </div>
37 + <div class="field-control">
38 + <input type="number" x-model.number="config.behaviour.timeout" placeholder="300" />
39 + </div>
40 + </div>
41 + <div class="field">
42 + <div class="field-label">
43 + <div class="field-title">Max tokens</div>
44 + <div class="field-description">Maximum output tokens for each delegated vision call.</div>
45 + </div>
46 + <div class="field-control">
47 + <input type="number" x-model.number="config.behaviour.max_tokens" placeholder="2000" />
48 + </div>
49 + </div>
50 + <div class="field" style="margin-top:1rem; opacity:0.7;">
51 + <div class="field-description">Pushback: <code>vision_load(..., raw=true)</code> always bypasses delegation and injects images directly into Main (useful for side-by-side comparison when Main must see pixels).</div>
52 + </div>
53 + </div>
54 +</body>
55 +</html>
plugins/_vision_sidecar/webui/thumbnail.jpg
Binary files /dev/null and b/plugins/_vision_sidecar/webui/thumbnail.jpg differ