@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / c994223d56
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 03:15 UTC

[Bugfix] update the prefix for qwen2 (#11795)

Co-authored-by: jiadi.jjd <jiadi.jjd@antgroup.com>

jiangjiadi committed Jan 8, 2025 at 02:36 UTC c994223d569221652643e897d8402b835ead411d
1 file changed +1 -1
vllm/model_executor/models/qwen2.py
+1 -1
@@ -298,7 +298,7 @@ class Qwen2Model(nn.Module):
298 lambda prefix: Qwen2DecoderLayer(config=config,
299 cache_config=cache_config,
300 quant_config=quant_config,
301 - prefix=f"{prefix}.layers"),
301 + prefix=prefix),
302 prefix=f"{prefix}.layers",
303 )
304