@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / a36de682d4
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 00:30 UTC

[Minor] Fix small typo in llama.py: QKVParallelLinear -> QuantizationConfig (#4991)

Philipp Moritz committed May 22, 2024 at 15:26 UTC a36de682d4283c60777bc3022ed3ce71cd90b904
1 file changed +1 -1
vllm/model_executor/models/llama.py
+1 -1
@@ -57,7 +57,7 @@ class LlamaMLP(nn.Module):
57 hidden_size: int,
58 intermediate_size: int,
59 hidden_act: str,
60 - quant_config: Optional[QKVParallelLinear] = None,
60 + quant_config: Optional[QuantizationConfig] = None,
61 bias: bool = False,
62 ) -> None:
63 super().__init__()