Default model transports to Chat Completions
Route OpenAI, Azure, GitHub Copilot, and GitHub Copilot OAuth explicitly through Chat Completions. Keep Codex/ChatGPT and xAI Grok OAuth explicitly on Responses. Make Chat Completions the LiteLLM transport default for omitted, blank, default, auto, or unknown modes so new providers avoid unsupported Responses endpoints unless they explicitly opt in.
Alessandro committed
Aug 27, 2026 at 12:40 UTC
dac15f77a336be4843f5b80ddd2326fb8fba7674
9 files changed
+49
-52
conf/AGENTS.md
+1
-1
@@ -16,7 +16,7 @@
16
- Do not commit API keys, provider secrets, local account identifiers, or private endpoints.
17
- Keep provider IDs and settings keys stable unless all loaders, UI references, migrations, and tests are updated.
18
- Defaults must work in a clean checkout and in Docker.
19
-- Providers without a native Responses path in the supported LiteLLM runtime, or intentionally standardized on Chat Completions, must set `a0_api_mode: chat`; native Responses providers rely on the Responses default.
19
+- Chat Completions is the transport default; providers intentionally using Responses must set `a0_api_mode: responses` explicitly.
20
- Templates must avoid accidentally unignoring private runtime content.
21
22
## Work Guidance
conf/model_providers.yaml
+5
@@ -71,6 +71,7 @@ chat:
71
name: GitHub Copilot
72
litellm_provider: github_copilot
73
kwargs:
74
+ a0_api_mode: chat
75
extra_headers:
76
"Editor-Version": "vscode/1.85.1"
77
"Copilot-Integration-Id": "vscode-chat"
@@ -180,6 +181,8 @@ chat:
181
litellm_provider: openai
182
models_list:
183
endpoint_url: "https://api.openai.com/v1/models"
184
+ kwargs:
185
+ a0_api_mode: chat
186
azure:
187
name: OpenAI Azure
188
litellm_provider: azure
@@ -187,6 +190,8 @@ chat:
190
endpoint_url: "/openai/models"
191
params:
192
api-version: "2024-10-21"
193
+ kwargs:
194
+ a0_api_mode: chat
195
bedrock:
196
name: AWS Bedrock
197
litellm_provider: bedrock
helpers/litellm_transport.py
+3
-12
@@ -35,14 +35,7 @@ class TransportRecovery(Enum):
35
FALLBACK_TO_CHAT = "fallback_to_chat"
36
37
38
-CHAT_COMPLETIONS_ALIASES = {
39
- "chat",
40
- "chat_completion",
41
- "chat_completions",
42
- "completion",
43
- "completions",
44
-}
45
-RESPONSES_ALIASES = {"", "auto", "default", "response", "responses", "responses_api"}
38
+RESPONSES_ALIASES = {"response", "responses", "responses_api"}
39
RESPONSES_REASONING_EFFORTS = {"minimal", "low", "medium", "high"}
40
RESPONSES_REASONING_FALLBACK_EFFORT = "high"
41
NO_REASONING_EFFORT_ALIASES = {"", "0", "false", "no", "none", "off", "disabled"}
@@ -146,12 +139,10 @@ class TransportPolicy:
139
140
@staticmethod
141
def _pop_mode(kwargs: dict[str, Any]) -> TransportMode:
149
- value = str(kwargs.pop("a0_api_mode", "responses") or "").lower().strip()
150
- if value in CHAT_COMPLETIONS_ALIASES:
151
- return TransportMode.CHAT_COMPLETIONS
142
+ value = str(kwargs.pop("a0_api_mode", "") or "").lower().strip()
143
if value in RESPONSES_ALIASES:
144
return TransportMode.RESPONSES
154
- return TransportMode.RESPONSES
145
+ return TransportMode.CHAT_COMPLETIONS
146
147
@property
148
def using_responses(self) -> bool:
helpers/litellm_transport.py.dox.md
+1
-1
@@ -27,7 +27,7 @@
27
- Do not send orphan tool controls when no tools are present; strict OpenAI-compatible servers can reject empty `tools` arrays.
28
- When Agent Zero function tools are present, default Responses requests to one required native call; explicit request-level `tool_choice` and `parallel_tool_calls` values still win.
29
- Normalize function tool parameter schemas with an explicit object `properties` field before Responses requests so OpenAI-compatible chat backends reached through LiteLLM can validate them.
30
-- Prefer Responses API when configured, but fallback to Chat Completions when the provider does not support Responses.
30
+- Default to Chat Completions; use Responses only when `a0_api_mode` explicitly selects it, with fallback to Chat Completions when unsupported.
31
- Fall back to Chat Completions when a Responses request is rejected before any output by an endpoint-specific or shape-specific Bad Request indicating the provider cannot parse Responses payloads.
32
- Treat opaque type-discrimination errors such as `cannot determine type` from OpenAI-compatible Responses endpoints as shape-specific rejections.
33
- Fall back to Chat Completions when a Responses endpoint fails before output with an endpoint-specific server error, proxy path-unavailable error, or LiteLLM proxy-extra import error.
plugins/_oauth/AGENTS.md
+1
-1
@@ -48,7 +48,7 @@
48
- Codex Responses proxy defaults for reasoning effort, reasoning summary, and text verbosity come from the `codex` plugin config; explicit native request values take precedence.
49
- Codex request shaping tightens an already-advertised native `response` tool to a strict required `text` schema; it must not add tools omitted by the framework tool policy.
50
- Non-streaming Codex proxy responses must retain completed SSE output items when the final `response.completed` envelope omits them.
51
-- OAuth providers without upstream Responses support must set `a0_api_mode: chat`; native Responses providers rely on the default, since a local proxy route alone does not prove upstream support.
51
+- OAuth providers intentionally using Responses must set `a0_api_mode: responses`; all others inherit the Chat Completions default, since a local proxy route alone does not prove upstream support.
52
53
## Work Guidance
54
plugins/_oauth/conf/model_providers.yaml
+3
@@ -6,6 +6,7 @@ chat:
6
models_list:
7
endpoint_url: "/models"
8
kwargs:
9
+ a0_api_mode: responses
10
api_base: "http://127.0.0.1/oauth/codex/v1"
11
github_copilot_oauth:
12
name: GitHub Copilot Account
@@ -14,6 +15,7 @@ chat:
15
models_list:
16
endpoint_url: "/models"
17
kwargs:
18
+ a0_api_mode: chat
19
api_base: "http://127.0.0.1/oauth/github-copilot/v1"
20
gemini_api_oauth:
21
name: Google Cloud Gemini Account
@@ -31,4 +33,5 @@ chat:
33
models_list:
34
endpoint_url: "/models"
35
kwargs:
36
+ a0_api_mode: responses
37
api_base: "http://127.0.0.1/oauth/xai-grok/v1"
tests/test_model_config_api_keys.py
+11
-31
@@ -531,39 +531,19 @@ def test_provider_api_mode_defaults_use_intended_transport():
531
).read_text(encoding="utf-8")
532
)
533
534
- chat_providers = (
535
- "anthropic",
536
- "cometapi",
537
- "deepseek",
538
- "google",
539
- "groq",
540
- "huggingface",
541
- "mistral",
542
- "moonshot",
543
- "nebius",
544
- "nvidia_nim",
545
- "bedrock",
546
- "openrouter",
547
- "sambanova",
548
- "xai",
549
- "zai",
550
- "zai_coding",
551
- )
552
- responses_providers = ("azure", "github_copilot", "openai")
553
-
554
- for provider in chat_providers:
555
- assert provider_config["chat"][provider]["kwargs"]["a0_api_mode"] == "chat"
556
-
557
- for provider in responses_providers:
558
- assert "a0_api_mode" not in provider_config["chat"][provider].get("kwargs", {})
534
+ for provider in provider_config["chat"].values():
535
+ assert provider.get("kwargs", {}).get("a0_api_mode", "chat") == "chat"
536
560
- assert (
561
- oauth_provider_config["chat"]["gemini_api_oauth"]["kwargs"]["a0_api_mode"]
562
- == "chat"
563
- )
537
+ responses_providers = {
538
+ provider
539
+ for provider, config in oauth_provider_config["chat"].items()
540
+ if config.get("kwargs", {}).get("a0_api_mode") == "responses"
541
+ }
542
+ assert responses_providers == {"codex_oauth", "xai_grok_oauth"}
543
565
- for provider in ("codex_oauth", "github_copilot_oauth", "xai_grok_oauth"):
566
- assert "a0_api_mode" not in oauth_provider_config["chat"][provider]["kwargs"]
544
+ for provider, config in oauth_provider_config["chat"].items():
545
+ if provider not in responses_providers:
546
+ assert config.get("kwargs", {}).get("a0_api_mode", "chat") == "chat"
547
548
549
def test_missing_api_key_banner_does_not_include_auto_modal_metadata(monkeypatch):
tests/test_responses_architecture.py
+11
-2
@@ -236,6 +236,7 @@ async def test_transport_retries_provider_state_as_local_replay(monkeypatch):
236
model="openai/gpt-5.4",
237
messages=[{"role": "user", "content": "new"}],
238
kwargs={
239
+ "a0_api_mode": "responses",
240
"previous_response_id": "resp_1",
241
"responses_input_items": [{"role": "user", "content": "new"}],
242
"responses_local_input_items": [{"role": "user", "content": "full"}],
@@ -276,7 +277,10 @@ async def test_transport_downgrades_unsupported_builtin_tools(monkeypatch):
277
transport = litellm_transport.LiteLLMTransport(
278
model="openai/gpt-5.4",
279
messages=[{"role": "user", "content": "new"}],
279
- kwargs={"responses_builtin_tools": [{"type": "web_search"}]},
280
+ kwargs={
281
+ "a0_api_mode": "responses",
282
+ "responses_builtin_tools": [{"type": "web_search"}],
283
+ },
284
)
285
286
parsed = await transport.acomplete()
@@ -291,7 +295,10 @@ async def test_transport_downgrades_unsupported_builtin_tools(monkeypatch):
295
next_transport = litellm_transport.LiteLLMTransport(
296
model="openai/gpt-5.4",
297
messages=[{"role": "user", "content": "again"}],
294
- kwargs={"responses_builtin_tools": [{"type": "web_search"}]},
298
+ kwargs={
299
+ "a0_api_mode": "responses",
300
+ "responses_builtin_tools": [{"type": "web_search"}],
301
+ },
302
)
303
request = next_transport._responses_request(stream=False)
304
assert "tools" not in request
@@ -344,6 +351,7 @@ async def test_unified_turn_keeps_streamed_call_when_completion_omits_output(
351
model="test-model",
352
provider="openai",
353
model_config=None,
354
+ a0_api_mode="responses",
355
)
356
357
async def response_callback(chunk: str, full: str):
@@ -430,6 +438,7 @@ async def test_unified_turn_waits_for_completed_native_responses_calls(monkeypat
438
model="test-model",
439
provider="openai",
440
model_config=None,
441
+ a0_api_mode="responses",
442
)
443
444
async def response_callback(chunk: str, full: str):
tests/test_stream_tool_early_stop.py
+13
-4
@@ -386,6 +386,7 @@ async def test_unified_call_closes_responses_stream_when_callback_raises(monkeyp
386
model="test-model",
387
provider="openai",
388
model_config=None,
389
+ a0_api_mode="responses",
390
)
391
392
async def response_callback(chunk: str, full: str):
@@ -401,7 +402,7 @@ async def test_unified_call_closes_responses_stream_when_callback_raises(monkeyp
402
403
404
@pytest.mark.asyncio
404
-async def test_chat_completions_escape_hatch_still_uses_acompletion(monkeypatch):
405
+async def test_chat_completions_default_uses_acompletion(monkeypatch):
406
stream = _AsyncChunkStream([_chunk("hello")])
407
calls: list[str] = []
408
@@ -425,7 +426,6 @@ async def test_chat_completions_escape_hatch_still_uses_acompletion(monkeypatch)
426
model="test-model",
427
provider="openai",
428
model_config=None,
428
- a0_api_mode="chat_completions",
429
)
430
431
async def response_callback(chunk: str, full: str):
@@ -508,6 +508,7 @@ async def test_unified_call_retries_responses_with_high_reasoning(monkeypatch):
508
model="gpt-5.4",
509
provider="openai",
510
model_config=None,
511
+ a0_api_mode="responses",
512
)
513
514
async def response_callback(chunk: str, full: str):
@@ -558,6 +559,7 @@ async def test_unified_call_falls_back_to_chat_when_responses_endpoint_missing(
559
model="claude-opus-4.7",
560
provider="openai",
561
model_config=None,
562
+ a0_api_mode="responses",
563
tool_choice="auto",
564
parallel_tool_calls=True,
565
)
@@ -616,6 +618,7 @@ async def test_unified_call_falls_back_when_litellm_hides_responses_404_url(
618
model="claude-opus-4.7",
619
provider="openai",
620
model_config=None,
621
+ a0_api_mode="responses",
622
)
623
624
async def response_callback(chunk: str, full: str):
@@ -672,6 +675,7 @@ async def test_unified_call_falls_back_for_proxy_responses_failures(
675
model="test-model",
676
provider="openai",
677
model_config=None,
678
+ a0_api_mode="responses",
679
)
680
681
async def response_callback(chunk: str, full: str):
@@ -720,6 +724,7 @@ async def test_unified_call_falls_back_when_responses_mock_reads_sse_as_json(
724
model="omniroute/test-model",
725
provider="openai",
726
model_config=None,
727
+ a0_api_mode="responses",
728
)
729
730
async def response_callback(chunk: str, full: str):
@@ -769,6 +774,7 @@ async def test_unified_call_falls_back_when_responses_bad_request_rejects_shape(
774
model="venice-model",
775
provider="openai",
776
model_config=None,
777
+ a0_api_mode="responses",
778
)
779
780
async def response_callback(chunk: str, full: str):
@@ -822,6 +828,7 @@ async def test_unified_call_raises_generic_responses_bad_request(monkeypatch):
828
model="test-model",
829
provider="openai",
830
model_config=None,
831
+ a0_api_mode="responses",
832
)
833
834
async def response_callback(chunk: str, full: str):
@@ -873,6 +880,7 @@ async def test_unified_call_preserves_cache_control_with_chat_for_non_native_res
880
model="claude-sonnet-4-5",
881
provider="anthropic",
882
model_config=None,
883
+ a0_api_mode="responses",
884
)
885
886
async def response_callback(chunk: str, full: str):
@@ -1157,6 +1165,7 @@ def test_complete_falls_back_to_chat_when_responses_shim_sends_empty_tools(
1165
model="hosted_vllm/qwen",
1166
messages=[{"role": "user", "content": "hi"}],
1167
kwargs={
1168
+ "a0_api_mode": "responses",
1169
"tools": [],
1170
"tool_choice": "auto",
1171
"parallel_tool_calls": True,
@@ -1358,12 +1367,12 @@ def test_cache_control_policy_keeps_native_responses_first():
1367
1368
openai_policy = litellm_transport.TransportPolicy.from_request(
1369
"openai/gpt-5.4",
1361
- {},
1370
+ {"a0_api_mode": "responses"},
1371
messages=messages,
1372
)
1373
anthropic_policy = litellm_transport.TransportPolicy.from_request(
1374
"anthropic/claude-sonnet-4-5",
1366
- {},
1375
+ {"a0_api_mode": "responses"},
1376
messages=messages,
1377
)
1378