A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 13:30 UTC
[Doc] small fix (#17277)
Signed-off-by: reidliu41 <reid201711@gmail.com> Co-authored-by: reidliu41 <reid201711@gmail.com>
Reid committed
Apr 28, 2025 at 11:53 UTC
f211331c48a28e4eb1f055048fa2fd33a07bc64d
2 files changed
+3
-1
docs/source/models/generative_models.md
+1
-1
@@ -59,7 +59,7 @@ A code example can be found here: <gh-file:examples/offline_inference/basic/basi
59
60
### `LLM.beam_search`
61
62
-The {class}`~vllm.LLM.beam_search` method implements [beam search](https://huggingface.co/docs/transformers/en/generation_strategies#beam-search-decoding) on top of {class}`~vllm.LLM.generate`.
62
+The {class}`~vllm.LLM.beam_search` method implements [beam search](https://huggingface.co/docs/transformers/en/generation_strategies#beam-search) on top of {class}`~vllm.LLM.generate`.
63
For example, to search using 5 beams and output at most 50 tokens:
64
65
```python
docs/source/models/supported_models.md
+2
@@ -793,6 +793,8 @@ or `--limit-mm-per-prompt` (online serving). For example, to enable passing up t
793
Offline inference:
794
795
```python
796
+from vllm import LLM
797
+
798
llm = LLM(
799
model="Qwen/Qwen2-VL-7B-Instruct",
800
limit_mm_per_prompt={"image": 4},