@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 913df14da3
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 15, 23:30 UTC

[Bugfix] Remove unused seq_group_metadata_list from ModelInputForGPU (#12935)

Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>

shangmingc committed Feb 8, 2025 at 22:46 UTC 913df14da3014d9432bfd8a5114f845ab567b1c6
1 file changed -1
vllm/worker/model_runner.py
-1
@@ -98,7 +98,6 @@ class ModelInputForGPU(ModelRunnerInputBase):
98 finished_requests_ids: Optional[List[str]] = None
99 virtual_engine: int = 0
100 async_callback: Optional[Callable] = None
101 - seq_group_metadata_list: Optional[List[SequenceGroupMetadata]] = None
101 scheduler_outputs: Optional[SchedulerOutputs] = None
102
103 def as_broadcastable_tensor_dict(self) -> Dict[str, Any]: