@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / ae495c74ea
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 00:30 UTC

[Doc]Replace deprecated flag in readme (#4526)

Ronen Schaffer committed May 30, 2024 at 01:26 UTC ae495c74eab390e52bcade098ee8313679fa8802
1 file changed +2 -1
examples/production_monitoring/README.md
+2 -1
@@ -29,7 +29,8 @@ python3 ../../benchmarks/benchmark_serving.py \
29 --model mistralai/Mistral-7B-v0.1 \
30 --tokenizer mistralai/Mistral-7B-v0.1 \
31 --endpoint /v1/completions \
32 - --dataset ShareGPT_V3_unfiltered_cleaned_split.json \
32 + --dataset-name sharegpt \
33 + --dataset-path ShareGPT_V3_unfiltered_cleaned_split.json \
34 --request-rate 3.0
35 ```
36