A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 02:15 UTC
[ROCm][Regression] Remove tensor creation that harms performance on ROCm (#20741)
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Gregory Shtrasberg committed
Jul 10, 2025 at 12:22 UTC
5b8366b61a49c74f161226fef3a8caa73900b02e
1 file changed
-4
vllm/platforms/rocm.py
-4
@@ -247,10 +247,6 @@ class RocmPlatform(Platform):
247
Set the device for the current platform.
248
"""
249
torch.cuda.set_device(device)
250
- # With this trick we can force the device to be set eagerly
251
- # see https://github.com/pytorch/pytorch/issues/155668
252
- # for why and when it is needed
253
- _ = torch.zeros(1, device=device)
250
251
@classmethod
252
@lru_cache(maxsize=8)