A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 4, 01:15 UTC
[Bugfix] Remove nested torch.compile in GDN rearrange_mixed_qkv causing CUDA graph capture failure (#42070)
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com> Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Thomas Parnell committed
May 9, 2026 at 17:30 UTC
3dda9aeb54cc15c9ecc6d1498a42ceb372d4472b
1 file changed
-1
vllm/model_executor/layers/mamba/gdn_linear_attn.py
-1
@@ -637,7 +637,6 @@ class GatedDeltaNetAttention(PluggableLayer, MambaBase):
637
638
return mixed_qkv_out, z_out, b_out, a_out
639
640
- @torch.compile(fullgraph=True)
640
def rearrange_mixed_qkv(self, mixed_qkv):
641
"""Split packed qkv into contiguous (1, seq, heads, dim) tensors.
642