@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 278be671a3
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 05:45 UTC

[Doc] Update model in arch_overview.rst to match comment (#10701)

Signed-off-by: spacewander <spacewanderlzx@gmail.com>

罗泽轩 committed Nov 28, 2024 at 15:58 UTC 278be671a355ea89843141928a426a303bfd8036
1 file changed +1 -1
docs/source/design/arch_overview.rst
+1 -1
@@ -42,7 +42,7 @@ Here is a sample of `LLM` class usage:
42 sampling_params = SamplingParams(temperature=0.8, top_p=0.95)
43
44 # Initialize the LLM engine with the OPT-125M model
45 - llm = LLM(model="Qwen/Qwen2.5-1.5B-Instruct")
45 + llm = LLM(model="facebook/opt-125m")
46
47 # Generate outputs for the input prompts
48 outputs = llm.generate(prompts, sampling_params)