A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 11, 08:45 UTC
[Doc][ResponsesAPI] add documentation (#30840)
Signed-off-by: Andrew Xia <axia@fb.com> Co-authored-by: Andrew Xia <axia@fb.com>
Andrew Xia committed
Dec 17, 2025 at 17:53 UTC
4c054d89aa5972014ba7e13c0accb0ab631b5638
2 files changed
+41
-4
docs/serving/openai_compatible_server.md
+27
@@ -47,6 +47,8 @@ We currently support the following OpenAI APIs:
47
- [Completions API](#completions-api) (`/v1/completions`)
48
- Only applicable to [text generation models](../models/generative_models.md).
49
- *Note: `suffix` parameter is not supported.*
50
+- [Responses API](#responses-api) (`/v1/responses`)
51
+ - Only applicable to [text generation models](../models/generative_models.md).
52
- [Chat Completions API](#chat-api) (`/v1/chat/completions`)
53
- Only applicable to [text generation models](../models/generative_models.md) with a [chat template](../serving/openai_compatible_server.md#chat-template).
54
- *Note: `user` parameter is ignored.*
@@ -229,6 +231,31 @@ The following extra parameters are supported:
231
--8<-- "vllm/entrypoints/openai/protocol.py:chat-completion-extra-params"
232
```
233
234
+### Responses API
235
+
236
+Our Responses API is compatible with [OpenAI's Responses API](https://platform.openai.com/docs/api-reference/responses);
237
+you can use the [official OpenAI Python client](https://github.com/openai/openai-python) to interact with it.
238
+
239
+Code example: [examples/online_serving/openai_responses_client_with_tools.py](../../examples/online_serving/openai_responses_client_with_tools.py)
240
+
241
+#### Extra parameters
242
+
243
+The following extra parameters in the request object are supported:
244
+
245
+??? code
246
+
247
+ ```python
248
+ --8<-- "vllm/entrypoints/openai/protocol.py:responses-extra-params"
249
+ ```
250
+
251
+The following extra parameters in the response object are supported:
252
+
253
+??? code
254
+
255
+ ```python
256
+ --8<-- "vllm/entrypoints/openai/protocol.py:responses-response-extra-params"
257
+ ```
258
+
259
### Embeddings API
260
261
Our Embeddings API is compatible with [OpenAI's Embeddings API](https://platform.openai.com/docs/api-reference/embeddings);
vllm/entrypoints/openai/protocol.py
+14
-4
@@ -1654,13 +1654,23 @@ class ResponsesResponse(OpenAIBaseModel):
1654
usage: ResponseUsage | None = None
1655
user: str | None = None
1656
1657
- # --8<-- [start:responses-extra-params]
1657
+ # --8<-- [start:responses-response-extra-params]
1658
# These are populated when enable_response_messages is set to True
1659
# NOTE: custom serialization is needed
1660
# see serialize_input_messages and serialize_output_messages
1661
- input_messages: ResponseInputOutputMessage | None = None
1662
- output_messages: ResponseInputOutputMessage | None = None
1663
- # --8<-- [end:responses-extra-params]
1661
+ input_messages: ResponseInputOutputMessage | None = Field(
1662
+ default=None,
1663
+ description=(
1664
+ "If enable_response_messages, we can show raw token input to model."
1665
+ ),
1666
+ )
1667
+ output_messages: ResponseInputOutputMessage | None = Field(
1668
+ default=None,
1669
+ description=(
1670
+ "If enable_response_messages, we can show raw token output of model."
1671
+ ),
1672
+ )
1673
+ # --8<-- [end:responses-response-extra-params]
1674
1675
# NOTE: openAI harmony doesn't serialize TextContent properly,
1676
# TODO: this fixes for TextContent, but need to verify for tools etc