A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 00:45 UTC
Fix bad path in prometheus example (#12481)
Signed-off-by: mgoin <michael@neuralmagic.com>
Michael Goin committed
Jan 27, 2025 at 20:56 UTC
426a5c362557c6df4604ed084660b8915fbca30c
1 file changed
+1
-1
examples/online_serving/prometheus_grafana/README.md
+1
-1
@@ -24,7 +24,7 @@ Submit some sample requests to the server:
24
```bash
25
wget https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered/resolve/main/ShareGPT_V3_unfiltered_cleaned_split.json
26
27
-python3 ../../benchmarks/benchmark_serving.py \
27
+python3 ../../../benchmarks/benchmark_serving.py \
28
--model mistralai/Mistral-7B-v0.1 \
29
--tokenizer mistralai/Mistral-7B-v0.1 \
30
--endpoint /v1/completions \