A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
Notice for deprecation of AutoAWQ (#26820)
Signed-off-by: HDCharles <39544797+HDCharles@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
HDCharles committed
Oct 14, 2025 at 16:39 UTC
b92ab3deda6aca2c0f05aba841f5edb05578af94
1 file changed
+4
docs/features/quantization/auto_awq.md
+4
@@ -1,5 +1,9 @@
1
# AutoAWQ
2
3
+> ⚠️ **Warning:**
4
+ The `AutoAWQ` library is deprecated. This functionality has been adopted by the vLLM project in [`llm-compressor`](https://github.com/vllm-project/llm-compressor/tree/main/examples/awq).
5
+ For the recommended quantization workflow, please see the AWQ examples in [`llm-compressor`](https://github.com/vllm-project/llm-compressor/tree/main/examples/awq). For more details on the deprecation, refer to the original [AutoAWQ repository](https://github.com/casper-hansen/AutoAWQ).
6
+
7
To create a new 4-bit quantized model, you can leverage [AutoAWQ](https://github.com/casper-hansen/AutoAWQ).
8
Quantization reduces the model's precision from BF16/FP16 to INT4 which effectively reduces the total model memory footprint.
9
The main benefits are lower latency and memory usage.