A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 10, 04:00 UTC
[Bugfix][ROCm][performance] Resolve the performance regression issue of the Qwen3-Next-80B-A3B-Thinking under rocm_atten (#32336)
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
vllmellm committed
Jan 15, 2026 at 03:32 UTC
e27078ea807cf5655c6be3b59608e55c1682756d
1 file changed
+10
-1
vllm/v1/attention/backends/rocm_attn.py
+10
-1
@@ -167,7 +167,16 @@ class RocmAttentionBackend(AttentionBackend):
167
# ROCM paged attention kernel only supports block sizes 16 and 32
168
# due to shared memory (LDS) constraints on AMD GPUs.
169
# See csrc/rocm/attention.cu CALL_CUSTOM_LAUNCHER_BLK macro.
170
- return [16, 32]
170
+
171
+ # However, The limitations in [16, 32] are reasonable for a native C++ kernel,
172
+ # but vLLM should allow support for non-standard sizes via the Triton path,
173
+ # as addressed in this PR: https://github.com/vllm-project/vllm/pull/31380,
174
+ # where the Triton kernel under rocm_atten does not support inference
175
+ # for a non-standard qwen3-next model with a block_size of 544.
176
+ # We have fixed the Triton kernel so that the standard model uses the original
177
+ # bit-addressing logic, while the non-standard model
178
+ # uses our optimized kernel logic.
179
+ return [16, 32, 544]
180
181
@classmethod
182
def get_supported_head_sizes(cls) -> list[int]: