A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 3, 05:30 UTC
[Doc] Sync CLI guide with actual help modes and launch subcommand (#40326)
Signed-off-by: Rui Wang <raygorous@gmail.com> Co-authored-by: Rui Wang <raygorous@gmail.com>
Ray Wang committed
May 20, 2026 at 02:32 UTC
87e31455b056c6ce59bf5dcb3c622155431851db
4 files changed
+46
-8
docs/cli/.nav.yml
+2
@@ -6,3 +6,5 @@ nav:
6
- run-batch.md
7
- vllm bench:
8
- bench/**/*.md
9
+ - vllm launch:
10
+ - launch/**/*.md
docs/cli/README.md
+20
-8
@@ -9,7 +9,7 @@ vllm --help
9
Available Commands:
10
11
```bash
12
-vllm {chat,complete,serve,bench,collect-env,run-batch}
12
+vllm {chat,complete,serve,launch,bench,collect-env,run-batch}
13
```
14
15
## serve
@@ -37,24 +37,36 @@ vllm serve meta-llama/Llama-2-7b-hf --uds /tmp/vllm.sock
37
Check with --help for more options:
38
39
```bash
40
-# To list all groups
41
-vllm serve --help=listgroup
40
+# To list all flags
41
+vllm serve --help=all
42
43
-# To view a argument group
43
+# To view an argument group
44
vllm serve --help=ModelConfig
45
46
# To view a single argument
47
vllm serve --help=max-num-seqs
48
49
-# To search by keyword
49
+# To search by keyword or flag name
50
vllm serve --help=max
51
-
52
-# To view full help with pager (less/more)
53
-vllm serve --help=page
51
```
52
53
See [vllm serve](./serve.md) for the full reference of all available arguments.
54
55
+## launch
56
+
57
+Launch individual vLLM components.
58
+
59
+```bash
60
+# Launch the rendering server component
61
+vllm launch render meta-llama/Llama-3.2-1B-Instruct
62
+
63
+# Inspect all available flags for the render component
64
+vllm launch render --help=all
65
+```
66
+
67
+See [vllm launch render](./launch/render.md) for the current launch
68
+component reference.
69
+
70
## chat
71
72
Generate chat completions via the running API server.
docs/cli/launch/render.md
new
+22
@@ -0,0 +1,22 @@
1
+# vllm launch render
2
+
3
+## Overview
4
+
5
+`vllm launch render` starts a GPU-less rendering server for preprocessing and
6
+postprocessing only.
7
+
8
+```bash
9
+vllm launch render meta-llama/Llama-3.2-1B-Instruct --port 8100
10
+```
11
+
12
+This command reuses the standard serving parser, so model, frontend,
13
+networking, and related CLI options follow the same conventions as
14
+[`vllm serve`](../serve.md).
15
+
16
+## JSON CLI Arguments
17
+
18
+--8<-- "docs/cli/json_tip.inc.md"
19
+
20
+## Arguments
21
+
22
+--8<-- "docs/generated/argparse/launch_render.inc.md"
docs/mkdocs/hooks/generate_argparse.py
+2
@@ -147,6 +147,7 @@ AsyncEngineArgs = auto_mock("vllm.engine.arg_utils", "AsyncEngineArgs")
147
EngineArgs = auto_mock("vllm.engine.arg_utils", "EngineArgs")
148
ChatCommand = auto_mock("vllm.entrypoints.cli.openai", "ChatCommand")
149
CompleteCommand = auto_mock("vllm.entrypoints.cli.openai", "CompleteCommand")
150
+RenderSubcommand = auto_mock("vllm.entrypoints.cli.launch", "RenderSubcommand")
151
openai_cli_args = auto_mock("vllm.entrypoints.openai", "cli_args")
152
openai_run_batch = auto_mock("vllm.entrypoints.openai", "run_batch")
153
@@ -260,6 +261,7 @@ def on_startup(command: Literal["build", "gh-deploy", "serve"], dirty: bool):
261
"serve": create_parser(openai_cli_args.make_arg_parser),
262
"chat": create_parser(ChatCommand.add_cli_args),
263
"complete": create_parser(CompleteCommand.add_cli_args),
264
+ "launch_render": create_parser(RenderSubcommand.add_cli_args),
265
"run-batch": create_parser(openai_run_batch.make_arg_parser),
266
# Benchmark CLI
267
"bench_latency": create_parser(bench_latency.add_cli_args),