@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / d79d9eaaff
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 00:30 UTC

[Misc] remove duplicate definition of `seq_lens_tensor` in model_runner.py (#5129)

Hyunsung Lee committed May 30, 2024 at 22:56 UTC d79d9eaaff90801668613a4e3d5d8a0004963f21
1 file changed -3
vllm/worker/model_runner.py
-3
@@ -518,9 +518,6 @@ class ModelRunner:
518 else:
519 multi_modal_input = None
520
521 - seq_lens_tensor = torch.tensor(seq_lens,
522 - dtype=torch.int,
523 - device=self.device)
521 query_lens_tensor = torch.tensor(query_lens,
522 dtype=torch.long,
523 device=self.device)