A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 05:45 UTC
[Doc] Update model in arch_overview.rst to match comment (#10701)
Signed-off-by: spacewander <spacewanderlzx@gmail.com>
罗泽轩 committed
Nov 28, 2024 at 15:58 UTC
278be671a355ea89843141928a426a303bfd8036
1 file changed
+1
-1
docs/source/design/arch_overview.rst
+1
-1
@@ -42,7 +42,7 @@ Here is a sample of `LLM` class usage:
42
sampling_params = SamplingParams(temperature=0.8, top_p=0.95)
43
44
# Initialize the LLM engine with the OPT-125M model
45
- llm = LLM(model="Qwen/Qwen2.5-1.5B-Instruct")
45
+ llm = LLM(model="facebook/opt-125m")
46
47
# Generate outputs for the input prompts
48
outputs = llm.generate(prompts, sampling_params)