A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 12:15 UTC
[ROCm] update installation guide to include build aiter from source instructions (#17542)
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Hongxia Yang committed
May 1, 2025 at 14:01 UTC
4acfa3354a2e100c88db99f4bcb5b5fc098211cb
1 file changed
+16
-1
docs/source/getting_started/installation/gpu/rocm.inc.md
+16
-1
@@ -73,7 +73,22 @@ Currently, there are no pre-built ROCm wheels.
73
You might need to downgrade the "ninja" version to 1.10 it is not used when compiling flash-attention-2 (e.g. `pip install ninja==1.10.2.4`)
74
:::
75
76
-3. Build vLLM. For example, vLLM on ROCM 6.3 can be built with the following steps:
76
+3. If you choose to build AITER yourself to use a certain branch or commit, you can build AITER using the following steps:
77
+
78
+ ```console
79
+ python3 -m pip uninstall -y aiter
80
+ git clone --recursive https://github.com/ROCm/aiter.git
81
+ cd aiter
82
+ git checkout $AITER_BRANCH_OR_COMMIT
83
+ git submodule sync; git submodule update --init --recursive
84
+ python3 setup.py develop
85
+ ```
86
+
87
+ :::{note}
88
+ You will need to config the `$AITER_BRANCH_OR_COMMIT` for your purpose.
89
+ :::
90
+
91
+4. Build vLLM. For example, vLLM on ROCM 6.3 can be built with the following steps:
92
93
```bash
94
$ pip install --upgrade pip