A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 08:30 UTC
[Misc] small improve (#18680)
Signed-off-by: reidliu41 <reid201711@gmail.com> Co-authored-by: reidliu41 <reid201711@gmail.com>
Reid committed
May 25, 2025 at 21:05 UTC
3a886bd58cecf3ce78c1eafed1f7d3d16800bccc
1 file changed
+1
-1
docs/features/quantization/bnb.md
+1
-1
@@ -15,7 +15,7 @@ pip install bitsandbytes>=0.45.3
15
16
vLLM reads the model's config file and supports both in-flight quantization and pre-quantized checkpoint.
17
18
-You can find bitsandbytes quantized models on <https://huggingface.co/models?search=bitsandbytes>.
18
+You can find bitsandbytes quantized models on [Hugging Face](https://huggingface.co/models?search=bitsandbytes).
19
And usually, these repositories have a config.json file that includes a quantization_config section.
20
21
## Read quantized checkpoint