@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 8a1415cf77
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 22:00 UTC

[Bugfix] GPTBigCodeForCausalLM: Remove lm_head from supported_lora_modules. (#6326)

Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com> Co-authored-by: Travis Johnson <tsjohnso@us.ibm.com>

Thomas Parnell committed Jul 11, 2024 at 16:05 UTC 8a1415cf776b2b902f6429ecfc325877b57cbefe
1 file changed +1 -1
vllm/model_executor/models/gpt_bigcode.py
+1 -1
@@ -235,7 +235,7 @@ class GPTBigCodeModel(nn.Module):
235 class GPTBigCodeForCausalLM(nn.Module, SupportsLoRA):
236 packed_modules_mapping = {"c_attn": ["c_attn"]}
237
238 - supported_lora_modules = ["c_fc", "c_proj", "wte", "lm_head", "c_attn"]
238 + supported_lora_modules = ["c_fc", "c_proj", "wte", "c_attn"]
239
240 embedding_modules = {
241 "wte": "input_embeddings",