A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 4, 01:15 UTC
[Docs] Update server entrypoint examples (#42077)
Signed-off-by: Ethan Feng <ethan.fengch@gmail.com>
Ethan Feng committed
May 9, 2026 at 10:03 UTC
a43bc34baf2a2e367cdbbdaec44de8cf5de92d1a
4 files changed
+5
-8
docs/deployment/frameworks/runpod.md
+1
-2
@@ -12,8 +12,7 @@ vLLM can be deployed on [RunPod](https://www.runpod.io/), a cloud GPU platform t
12
SSH into your RunPod pod and launch the vLLM OpenAI-compatible server:
13
14
```bash
15
-python -m vllm.entrypoints.openai.api_server \
16
- --model <model-name> \
15
+vllm serve <model-name> \
16
--host 0.0.0.0 \
17
--port 8000
18
```
docs/deployment/integrations/kthena.md
+2
-3
@@ -79,9 +79,8 @@ Key points from the example YAML:
79
- -c
80
- >
81
bash /vllm-workspace/examples/ray_serving/multi-node-serving.sh leader --ray_cluster_size=2;
82
- python3 -m vllm.entrypoints.openai.api_server
82
+ vllm serve meta-llama/Llama-3.1-405B-Instruct
83
--port 8080
84
- --model meta-llama/Llama-3.1-405B-Instruct
84
--tensor-parallel-size 8
85
--pipeline-parallel-size 2
86
```
@@ -145,7 +144,7 @@ spec:
144
- sh
145
- -c
146
- "bash /vllm-workspace/examples/ray_serving/multi-node-serving.sh leader --ray_cluster_size=2;
148
- python3 -m vllm.entrypoints.openai.api_server --port 8080 --model meta-llama/Llama-3.1-405B-Instruct --tensor-parallel-size 8 --pipeline-parallel-size 2"
147
+ vllm serve meta-llama/Llama-3.1-405B-Instruct --port 8080 --tensor-parallel-size 8 --pipeline-parallel-size 2"
148
resources:
149
limits:
150
nvidia.com/gpu: "8"
docs/design/lora_resolver_plugins.md
+1
-2
@@ -62,8 +62,7 @@ The filesystem resolver is installed with vLLM by default and enables loading Lo
62
3. **Start vLLM server**:
63
Your base model can be `meta-llama/Llama-2-7b-hf`. Please make sure you set up the Hugging Face token in your env var `export HF_TOKEN=xxx235`.
64
```bash
65
- python -m vllm.entrypoints.openai.api_server \
66
- --model your-base-model \
65
+ vllm serve your-base-model \
66
--enable-lora
67
```
68
docs/design/optimization_levels.md
+1
-1
@@ -16,7 +16,7 @@ User-set flags take precedence over optimization level defaults.
16
17
```bash
18
# CLI usage
19
-python -m vllm.entrypoints.api_server --model RedHatAI/Llama-3.2-1B-FP8 -O1
19
+vllm serve RedHatAI/Llama-3.2-1B-FP8 -O1
20
21
# Python API usage
22
from vllm.entrypoints.llm import LLM