A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 3, 00:30 UTC
[RFC][EPLB][#32028] Remove dead torch.accelerator.synchronize() from sync path (#40733)
Signed-off-by: SandishKumarHN <3078999+SandishKumarHN@users.noreply.github.com> Co-authored-by: SandishKumarHN <3078999+SandishKumarHN@users.noreply.github.com>
SandishKumarHN committed
May 22, 2026 at 19:19 UTC
977703aa94fa946e19a70de4c105625600255b66
1 file changed
-4
vllm/distributed/eplb/rebalance_execute.py
-4
@@ -584,10 +584,6 @@ def rearrange_expert_weights_inplace(
584
# have the same shape.
585
weights_buffer = [torch.empty_like(w) for w in first_layer_weights]
586
587
- # NOTE(bowen): We need this synchronize to run, but I don't know why.
588
- # If you figure out the reason, please let me know -- thank you!
589
- torch.accelerator.synchronize()
590
-
587
old_global_expert_indices_cpu = old_global_expert_indices.cpu().numpy()
588
new_global_expert_indices_cpu = new_global_expert_indices.cpu().numpy()
589