@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / cf8849f2d6
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 06:45 UTC

Add `MptForCausalLM` key in model_loader (#1526)

Wenfei Yan committed Oct 31, 2023 at 15:46 UTC cf8849f2d6ea0a6ab76c94d2f585e02aea9e1512
1 file changed +2
vllm/model_executor/model_loader.py
+2
@@ -27,6 +27,8 @@ _MODEL_REGISTRY = {
27 "LlamaForCausalLM": LlamaForCausalLM,
28 "LLaMAForCausalLM": LlamaForCausalLM, # For decapoda-research/llama-*
29 "MistralForCausalLM": MistralForCausalLM,
30 + # transformers's mpt class has lower case
31 + "MptForCausalLM": MPTForCausalLM,
32 "MPTForCausalLM": MPTForCausalLM,
33 "OPTForCausalLM": OPTForCausalLM,
34 "QWenLMHeadModel": QWenLMHeadModel,