| 1 | # history.py DOX |
| 2 | |
| 3 | ## Purpose |
| 4 | |
| 5 | - Own the `history.py` helper module. |
| 6 | - This module owns chat history message records and model-output conversion. |
| 7 | - Keep this file-level DOX profile synchronized with `history.py` because this directory is intentionally flat. |
| 8 | |
| 9 | ## Ownership |
| 10 | |
| 11 | - `history.py` owns the runtime implementation. |
| 12 | - `history.py.dox.md` owns durable notes about responsibilities, contracts, side effects, and verification for that implementation. |
| 13 | - Classes: |
| 14 | - `RawMessage` (`TypedDict`) |
| 15 | - `OutputMessage` (`TypedDict`) |
| 16 | - `Record` (no explicit base class) |
| 17 | - `get_tokens(self) -> int` |
| 18 | - `async compress(self) -> bool` |
| 19 | - `output(self) -> list[OutputMessage]` |
| 20 | - `async summarize(self) -> str` |
| 21 | - `to_dict(self) -> dict` |
| 22 | - `from_dict(data: dict, history: 'History')` |
| 23 | - `output_langchain(self)` |
| 24 | - `output_text(self, human_label=..., ai_label=...)` |
| 25 | - `Message` (`Record`) |
| 26 | - `get_tokens(self) -> int` |
| 27 | - `calculate_tokens(self)` |
| 28 | - `set_summary(self, summary: str)` |
| 29 | - `async compress(self)` |
| 30 | - `output(self)` |
| 31 | - `output_langchain(self)` |
| 32 | - `output_text(self, human_label=..., ai_label=...)` |
| 33 | - `to_dict(self)` |
| 34 | - `Topic` (`Record`) |
| 35 | - `get_tokens(self)` |
| 36 | - `add_message(self, ai: bool, content: MessageContent, tokens: int=..., id: str=...) -> Message` |
| 37 | - `output(self) -> list[OutputMessage]` |
| 38 | - `async summarize(self)` |
| 39 | - `compress_large_messages(self, message_ratio: float=...) -> bool` |
| 40 | - `async compress(self) -> bool` |
| 41 | - `async compress_attention(self, ratio: float=...) -> bool` |
| 42 | - `async summarize_messages(self, messages: list[Message])` |
| 43 | - `Bulk` (`Record`) |
| 44 | - `get_tokens(self)` |
| 45 | - `output(self, human_label: str=..., ai_label: str=...) -> list[OutputMessage]` |
| 46 | - `async compress(self)` |
| 47 | - `async summarize(self)` |
| 48 | - `to_dict(self)` |
| 49 | - `from_dict(data: dict, history: 'History')` |
| 50 | - `History` (`Record`) |
| 51 | - `get_tokens(self) -> int` |
| 52 | - `is_over_limit(self)` |
| 53 | - `get_bulks_tokens(self) -> int` |
| 54 | - `get_topics_tokens(self) -> int` |
| 55 | - `get_current_topic_tokens(self) -> int` |
| 56 | - `add_message(self, ai: bool, content: MessageContent, tokens: int=..., id: str=...) -> Message` |
| 57 | - `new_topic(self)` |
| 58 | - `output(self) -> list[OutputMessage]` |
| 59 | - Top-level functions: |
| 60 | - `deserialize_history(json_data: str, agent) -> History` |
| 61 | - `_stringify_output(output: OutputMessage, ai_label=..., human_label=...)` |
| 62 | - `_stringify_content(content: MessageContent) -> str` |
| 63 | - `_output_content_langchain(content: MessageContent)` |
| 64 | - `group_outputs_abab(outputs: list[OutputMessage]) -> list[OutputMessage]` |
| 65 | - `group_messages_abab(messages: list[BaseMessage]) -> list[BaseMessage]` |
| 66 | - `output_langchain(messages: list[OutputMessage])` |
| 67 | - `output_text(messages: list[OutputMessage], ai_label=..., human_label=...)` |
| 68 | - `clear_responses_provider_state(agent) -> None` |
| 69 | - `_merge_outputs(a: MessageContent, b: MessageContent) -> MessageContent` |
| 70 | - `_merge_properties(a: Dict[str, MessageContent], b: Dict[str, MessageContent]) -> Dict[str, MessageContent]` |
| 71 | - `_is_raw_message(obj: object) -> bool` |
| 72 | - `_is_embedded_data(obj: object) -> bool` |
| 73 | - `_json_dumps(obj)` |
| 74 | - `_json_loads(obj)` |
| 75 | - Notable constants/configuration names: `BULK_MERGE_COUNT`, `TOPICS_MERGE_COUNT`, `CURRENT_TOPIC_RATIO`, `HISTORY_TOPIC_RATIO`, `HISTORY_BULK_RATIO`, `CURRENT_TOPIC_ATTENTION_COMPRESSION`, `HISTORY_TOPIC_ATTENTION_COMPRESSION`, `LARGE_MESSAGE_TO_CURRENT_TOPIC_RATIO`, `LARGE_MESSAGE_TO_HISTORY_TOPIC_RATIO`, `RAW_MESSAGE_OUTPUT_TEXT_TRIM`, `COMPRESSION_TARGET_RATIO`. |
| 76 | |
| 77 | ## Runtime Contracts |
| 78 | |
| 79 | - Helper modules own reusable framework APIs and must preserve public callers unless all callers, tests, and docs are updated together. |
| 80 | - Update this file whenever public functions, classes, persistence behavior, path/security assumptions, side effects, or cross-module contracts change. |
| 81 | - `clear_responses_provider_state(agent)` removes the active provider continuation IDs after local history rewrites while preserving stored response ID lists for later cleanup. |
| 82 | - `Message.from_dict()` normalizes legacy AI Responses metadata through `LLMResult.metadata()` so loaded chats shed transient payloads while unrelated metadata and non-AI tool-result inputs remain intact. |
| 83 | - `output_langchain()` removes leading assistant messages after grouping so provider histories always begin with a user turn; the WebUI greeting remains persisted and displayed but is not sent as an orphaned assistant message. |
| 84 | - `_json_dumps()` emits compact JSON (`","`, `":"` separators) for serialized history and generated non-string user-turn content. |
| 85 | - Observed side-effect areas: filesystem writes, filesystem deletion, model calls, plugin state, settings/state persistence, secret handling. |
| 86 | - Imported dependency areas include: `abc`, `asyncio`, `collections`, `collections.abc`, `enum`, `helpers`, `json`, `langchain_core.messages`, `math`, `plugins._model_config.helpers.model_config`, `typing`, `uuid`. |
| 87 | |
| 88 | ## Key Concepts |
| 89 | |
| 90 | - Important called helpers/classes observed in the source: `History`, `_is_raw_message`, `_json_dumps`, `group_messages_abab`, `join`, `make_list`, `cast`, `a.copy`, `json.dumps`, `json.loads`, `globals.from_dict`, `output_langchain`, `output_text`, `self.output_text`, `tokens.approximate_tokens`, `self.calculate_tokens`, `Message`, `get_chat_model_config`, `large_msgs.sort`, `self.compress_large_messages`. |
| 91 | - Keep request/response, tool, or helper semantics documented here at the same time as source changes. |
| 92 | |
| 93 | ## Work Guidance |
| 94 | |
| 95 | - Preserve public helper APIs used by core code and plugins unless every caller is updated. |
| 96 | - Keep path, auth, secret, persistence, network, and subprocess behavior explicit and bounded. |
| 97 | - Prefer adding cohesive helper functions here only when behavior is reused across modules. |
| 98 | |
| 99 | ## Verification |
| 100 | |
| 101 | - Run targeted tests for changed helper behavior; run security regressions for auth, filesystem, WebSocket, tunnel, upload, or secret-handling helpers. |
| 102 | - Related tests observed by source search: |
| 103 | - `tests/test_browser_agent_regressions.py` |
| 104 | - `tests/test_chat_compaction.py` |
| 105 | - `tests/test_error_retry_plugin.py` |
| 106 | - `tests/test_history_compression_wait.py` |
| 107 | - `tests/test_mcp_handler_multimodal.py` |
| 108 | - `tests/test_memory_quality.py` |
| 109 | - `tests/test_model_config_project_presets.py` |
| 110 | - `tests/test_office_document_store.py` |
| 111 | |
| 112 | ## Child DOX Index |
| 113 | |
| 114 | No child DOX files. |