@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 4a6769053a
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 23:15 UTC

[CI][BugFix] Flip is_quant_method_supported condition (#5577)

Michael Goin committed Jun 16, 2024 at 10:07 UTC 4a6769053ab2616f7f490e6ec5b8241e76ef0c2a
1 file changed +1 -1
tests/quantization/utils.py
+1 -1
@@ -10,5 +10,5 @@ def is_quant_method_supported(quant_method: str) -> bool:
10
11 capability = torch.cuda.get_device_capability()
12 capability = capability[0] * 10 + capability[1]
13 - return (capability <
13 + return (capability >=
14 QUANTIZATION_METHODS[quant_method].get_min_capability())