[Doc] Show how to get Responses prompt token IDs (#55912)
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> Co-authored-by: OpenAI Codex <codex@openai.com>
Francisco Javier Arceo committed
Sep 15, 2026 at 22:48 UTC
7d4bba54985589176a60cbf3ad4424be3ee84999
3 files changed
+44
-3
docs/serving/online_serving/README.md
+6
-1
@@ -130,7 +130,12 @@ Scale-out APIs are disabled by default on `vllm serve`. Set `--enable-scale-out`
130
131
### Renderer APIs
132
133
-For further details on renderer APIs, please refer to [this page](renderer.md).
133
+Renderer APIs preprocess completion, chat, and Responses requests without running
134
+inference. They handle tokenization, model-specific prompt formatting, and
135
+multimodal preprocessing, returning prompt token IDs, sampling parameters, and
136
+any processed multimodal inputs for generation.
137
+
138
+See the [renderer guide](renderer.md) for setup instructions and examples.
139
140
- [Completions Render API](renderer.md) (`/v1/completions/render`)
141
- Render completion requests
docs/serving/online_serving/openai_compatible_server.md
+5
@@ -199,6 +199,11 @@ you can use the [official OpenAI Python client](https://github.com/openai/openai
199
200
Code example: [examples/tool_calling/openai_responses_client_with_tools.py](../../../examples/tool_calling/openai_responses_client_with_tools.py)
201
202
+To get prompt token IDs before generation, use
203
+[`/v1/responses/render`](renderer.md#get-responses-prompt-token-ids). It preprocesses
204
+a self-contained Responses request without inference and returns the prompt
205
+token IDs, so a separate `/tokenize` call is not needed.
206
+
207
#### Extra parameters
208
209
The following extra parameters in the request object are supported:
docs/serving/online_serving/renderer.md
+33
-2
@@ -26,27 +26,58 @@ vllm serve <model> --enable-scale-out
26
- [Responses Render API](renderer.md) (`/v1/responses/render`)
27
- Render a self-contained Responses request
28
29
+## Get Responses prompt token IDs
30
+
31
+Use `/v1/responses/render` to get prompt token IDs before choosing a model
32
+replica. Rendering applies prompt construction and preprocessing without running
33
+inference.
34
+
35
The Responses render endpoint uses the same prompt construction as
36
`/v1/responses` and returns one token-in `GenerateRequest`. It is stateless:
37
inline history is supported, but `previous_response_id` is not. Callers must
38
resolve stored response state and include the resulting history in the request
39
before rendering.
40
41
+Configure the renderer and generation workers with the same model, tokenizer,
42
+chat template, and preprocessing options. Render the full request, including
43
+instructions, history, tools, and any template or truncation options, so the
44
+returned IDs reflect the prompt the model will receive.
45
+
46
For multimodal requests, the `GenerateRequest` contains the model-processed
47
multimodal payload, which can be substantially larger than the source image or
48
video. The caller must forward that payload unchanged to the generation service
49
and provision transport limits and memory accordingly.
50
51
+For example, start a standard inference server with scale-out endpoints enabled:
52
+
53
+```bash
54
+VLLM_ENABLE_SCALE_OUT_ENDPOINTS=1 \
55
+ vllm serve meta-llama/Llama-3.1-8B-Instruct
56
+```
57
+
58
+Send a Responses request and use `jq` to extract the `token_ids` field:
59
+
60
```bash
41
-curl http://localhost:8000/v1/responses/render \
61
+curl --fail --silent --show-error http://localhost:8000/v1/responses/render \
62
-H "Content-Type: application/json" \
63
-d '{
64
"model": "meta-llama/Llama-3.1-8B-Instruct",
65
"input": "Explain prefix caching in one sentence.",
66
"max_output_tokens": 32
47
- }'
67
+ }' | jq '.token_ids'
68
```
69
70
+To count the rendered prompt tokens for this text request, replace the `jq`
71
+filter with `'.token_ids | length'`. The endpoint returns the full
72
+`GenerateRequest`; `jq` filters the response on the client. Keep the complete
73
+response if you will forward it to `/inference/v1/generate`, including any
74
+multimodal features.
75
+
76
+If the server has `--api-key` or `VLLM_API_KEY` configured, add
77
+`-H "Authorization: Bearer <api-key>"` to the request. See
78
+[API key authentication limitations](../../usage/security.md#api-key-authentication-limitations)
79
+for the existing authentication boundaries.
80
+
81
For the post processing counterpart that turns generated token IDs back into OpenAI compatible responses, see the [Derenderer APIs](derenderer.md).
82
83
## Multimodal Render Features