A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 07:15 UTC
[Nit][Benchmark]Fix example in benchmark_serving_structured_output.py (#19311)
Signed-off-by: Lifan Shen <lifans@meta.com>
Lifans committed
Jun 7, 2025 at 03:25 UTC
4e4f63ad45e403613b7841ff1a0a1dd3b57fe836
1 file changed
-1
benchmarks/benchmark_serving_structured_output.py
-1
@@ -12,7 +12,6 @@ On the client side, run:
12
--model <your_model> \
13
--dataset json \
14
--structured-output-ratio 1.0 \
15
- --structured-output-backend auto \
15
--request-rate 10 \
16
--num-prompts 1000
17