@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / a480939e8e
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 17:00 UTC

[Bugfix] Fix weight loading issue by rename variable. (#8293)

Wenxiang committed Sep 13, 2024 at 07:25 UTC a480939e8e3b8e5b5571531c30212a1a947ee32e
1 file changed +1 -1
vllm/model_executor/models/phimoe.py
+1 -1
@@ -600,7 +600,7 @@ class PhiMoEForCausalLM(nn.Module, SupportsLoRA):
600 weight_loader(
601 param,
602 loaded_weight,
603 - weight_name,
603 + name,
604 shard_id=shard_id,
605 expert_id=expert_id,
606 )