| 1 | # litellm_transport.py DOX |
| 2 | |
| 3 | ## Purpose |
| 4 | |
| 5 | - Own Agent Zero's LiteLLM transport adapter for Chat Completions and Responses API calls. |
| 6 | - Normalize Agent Zero model-call kwargs into provider-safe LiteLLM requests. |
| 7 | - Preserve canonical response metadata for history, provider-state continuation, and fallback decisions. |
| 8 | |
| 9 | ## Ownership |
| 10 | |
| 11 | - `litellm_transport.py` owns the runtime implementation. |
| 12 | - `litellm_transport.py.dox.md` owns durable notes about responsibilities, contracts, side effects, and verification for that implementation. |
| 13 | - Classes: |
| 14 | - `TransportMode` |
| 15 | - `TransportRecovery` |
| 16 | - `TransportPolicy` |
| 17 | - `LiteLLMTransport` |
| 18 | - `ChatCompletionsTransport` |
| 19 | - `ResponsesTransport` |
| 20 | - `ResponsesEventParser` |
| 21 | - Top-level functions include transport cache reset, request normalization, parsing, prompt-cache preparation, and response/error classifiers. |
| 22 | |
| 23 | ## Runtime Contracts |
| 24 | |
| 25 | - Keep provider selection and provider-specific defaults outside this helper; callers pass a resolved LiteLLM model name and kwargs. |
| 26 | - Strip Agent Zero internal kwargs before sending requests to LiteLLM. |
| 27 | - Do not send orphan tool controls when no tools are present; strict OpenAI-compatible servers can reject empty `tools` arrays. |
| 28 | - When Agent Zero function tools are present, default Responses requests to one required native call; explicit request-level `tool_choice` and `parallel_tool_calls` values still win. |
| 29 | - Normalize function tool parameter schemas with an explicit object `properties` field before Responses requests so OpenAI-compatible chat backends reached through LiteLLM can validate them. |
| 30 | - Default to Chat Completions; use Responses only when `a0_api_mode` explicitly selects it, with fallback to Chat Completions when unsupported. |
| 31 | - Fall back to Chat Completions when a Responses request is rejected before any output by an endpoint-specific or shape-specific Bad Request indicating the provider cannot parse Responses payloads. |
| 32 | - Treat opaque type-discrimination errors such as `cannot determine type` from OpenAI-compatible Responses endpoints as shape-specific rejections. |
| 33 | - Fall back to Chat Completions when a Responses endpoint fails before output with an endpoint-specific server error, proxy path-unavailable error, or LiteLLM proxy-extra import error. |
| 34 | - Fall back to Chat Completions when LiteLLM's Responses mock streaming path tries to JSON-decode a real SSE stream before any output. |
| 35 | - Preserve Chat Completions tool calls from both non-streaming responses and streaming deltas as canonical `LLMResult` function-call items. |
| 36 | - Preserve provider usage and LiteLLM response cost for both transports only when the response or stream actually supplies them; do not synthesize unavailable provider accounting. |
| 37 | - Preserve Responses function calls collected from stream events when a terminal completed event omits them. |
| 38 | - Stream native `response` function arguments through a canonical response-tool envelope while continuing to buffer other function calls until completion. |
| 39 | - Serialize synthesized Responses function-call JSON with literal Unicode so streamed raw-response logs preserve tool arguments. |
| 40 | - Preserve provider-state metadata when Responses API calls succeed, and fall back to local replay when provider state is unsupported. |
| 41 | - Keep prompt-cache markers only for providers that accept them. |
| 42 | |
| 43 | ## Work Guidance |
| 44 | |
| 45 | - Add provider-agnostic request cleanup here when multiple OpenAI-compatible providers can benefit. |
| 46 | - Treat fallback behavior as a shared transport contract, not a provider registry. |
| 47 | - Keep tool conversion symmetric between Chat Completions and Responses requests. |
| 48 | |
| 49 | ## Verification |
| 50 | |
| 51 | - Run `pytest tests/test_stream_tool_early_stop.py tests/test_responses_architecture.py -q` after changing transport normalization or fallback behavior. |
| 52 | - Run local-provider smoke checks when changing OpenAI-compatible request cleanup. |
| 53 | |
| 54 | ## Child DOX Index |
| 55 | |
| 56 | No child DOX files. |