@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / ab9e8488d5
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 06:45 UTC

Add Yi model to quantization support (#1600)

forpanyang committed Nov 10, 2023 at 03:47 UTC ab9e8488d57687d1144e7b4802d195f35882ecdc
1 file changed +1
vllm/model_executor/model_loader.py
+1
@@ -41,6 +41,7 @@ _MODEL_REGISTRY = {
41 _MODEL_CLASSES_SUPPORT_QUANTIZATION = [
42 LlamaForCausalLM,
43 MistralForCausalLM,
44 + YiForCausalLM,
45 ]
46
47