@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 2838d6b38e
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 10:45 UTC

[Bugfix] Weight loading fix for OPT model (#9042)

Co-authored-by: dvres <dvres@fri.uni-lj.si>

Domen Vreš committed Oct 4, 2024 at 01:53 UTC 2838d6b38e1e37b303b01f2af0a9ddee2dd66f39
1 file changed +1 -1
vllm/model_executor/models/opt.py
+1 -1
@@ -353,7 +353,7 @@ class OPTForCausalLM(nn.Module):
353 ]
354 params_dict = dict(self.named_parameters(remove_duplicate=False))
355 for name, loaded_weight in weights:
356 - if "lm_head.weight" in name:
356 + if "lm_head.weight" in name and self.config.tie_word_embeddings:
357 continue
358 if name.startswith("decoder."):
359 name = "model." + name