A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[Bugfix][Qwen] fixes the weights dtype in qwen3_next: it is actually a bfloat16 (#27030)
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
Tao He committed
Oct 17, 2025 at 11:37 UTC
bde9e2272a28342d30fe9c4de4c1c9633ff153d8
1 file changed
-1
vllm/model_executor/models/qwen3_next.py
-1
@@ -325,7 +325,6 @@ class Qwen3NextGatedDeltaNet(nn.Module, MambaBase):
325
self.A_log = nn.Parameter(
326
torch.empty(
327
divide(self.num_v_heads, self.tp_size),
328
- dtype=torch.float32,
328
)
329
)
330