A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[Misc] Fix the benchmark's README and improve the error messages for the benchmark's argument checks (#22654)
Signed-off-by: tanruixiang <tanruixiang0104@gmail.com>
Ruixiang Tan committed
Aug 20, 2025 at 01:18 UTC
03d4235fd239f62393909080a7d63d4a03739051
2 files changed
+6
-2
benchmarks/README.md
+3
@@ -194,6 +194,7 @@ vllm serve Qwen/Qwen2-VL-7B-Instruct
194
```bash
195
vllm bench serve \
196
--backend openai-chat \
197
+ --endpoint-type openai-chat \
198
--model Qwen/Qwen2-VL-7B-Instruct \
199
--endpoint /v1/chat/completions \
200
--dataset-name hf \
@@ -230,6 +231,7 @@ vllm serve Qwen/Qwen2-VL-7B-Instruct
231
```bash
232
vllm bench serve \
233
--backend openai-chat \
234
+ --endpoint-type openai-chat \
235
--model Qwen/Qwen2-VL-7B-Instruct \
236
--endpoint /v1/chat/completions \
237
--dataset-name hf \
@@ -244,6 +246,7 @@ vllm bench serve \
246
```bash
247
vllm bench serve \
248
--backend openai-chat \
249
+ --endpoint-type openai-chat \
250
--model Qwen/Qwen2-VL-7B-Instruct \
251
--endpoint /v1/chat/completions \
252
--dataset-name hf \
vllm/benchmarks/datasets.py
+3
-2
@@ -740,10 +740,11 @@ def get_samples(args, tokenizer) -> list[SampleRequest]:
740
"openai-chat",
741
"openai-audio",
742
]:
743
- # multi-modal benchmark is only available on OpenAI Chat backend.
743
+ # multi-modal benchmark is only available on OpenAI Chat
744
+ # endpoint-type.
745
raise ValueError(
746
"Multi-modal content is only supported on 'openai-chat' and "
746
- "'openai-audio' backend.")
747
+ "'openai-audio' endpoint-type.")
748
input_requests = dataset_class(
749
dataset_path=args.dataset_path,
750
dataset_subset=args.hf_subset,