A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 2, 18:15 UTC
[Doc] Add Ascend NPU tab to the quickstart installation guide (#43550)
Signed-off-by: Aditya Singh <adisin650@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Aditya Singh committed
May 27, 2026 at 01:41 UTC
ad464e16c00ac115c9b9e2ae33521931b2f90966
2 files changed
+11
docs/getting_started/quickstart.md
+9
@@ -76,6 +76,15 @@ This guide will help you quickly get started with vLLM to perform:
76
!!! note
77
For more detailed instructions, including Docker, installing from source, and troubleshooting, please refer to the [vLLM on TPU documentation](https://docs.vllm.ai/projects/tpu/en/latest/).
78
79
+=== "Ascend NPU"
80
+
81
+ If you are using Ascend NPUs, you can run vLLM through [vLLM Ascend](https://github.com/vllm-project/vllm-ascend), a community-maintained hardware plugin.
82
+
83
+ Follow the installation instructions in the [vLLM Ascend quick start](https://docs.vllm.ai/projects/ascend/en/latest/quick_start.html).
84
+
85
+ !!! note
86
+ Ascend setup depends on your NPU hardware and CANN version. For supported versions, Docker images, and troubleshooting, please refer to the [vLLM Ascend documentation](https://docs.vllm.ai/projects/ascend/en/latest/).
87
+
88
=== "Apple Silicon (Mac)"
89
90
If you are using Apple Silicon Macs, you can use vLLM-Metal for GPU-accelerated inference via Apple's Metal framework.
pyproject.toml
+2
@@ -181,6 +181,8 @@ VALU = "VALU"
181
# Walsh-Hadamard Transform
182
wht = "wht"
183
WHT = "WHT"
184
+# Huawei Compute Architecture for Neural Networks
185
+CANN = "CANN"
186
187
[tool.uv]
188
no-build-isolation-package = ["torch"]