A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 00:30 UTC
[Doc]Replace deprecated flag in readme (#4526)
Ronen Schaffer committed
May 30, 2024 at 01:26 UTC
ae495c74eab390e52bcade098ee8313679fa8802
1 file changed
+2
-1
examples/production_monitoring/README.md
+2
-1
@@ -29,7 +29,8 @@ python3 ../../benchmarks/benchmark_serving.py \
29
--model mistralai/Mistral-7B-v0.1 \
30
--tokenizer mistralai/Mistral-7B-v0.1 \
31
--endpoint /v1/completions \
32
- --dataset ShareGPT_V3_unfiltered_cleaned_split.json \
32
+ --dataset-name sharegpt \
33
+ --dataset-path ShareGPT_V3_unfiltered_cleaned_split.json \
34
--request-rate 3.0
35
```
36