@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 4b5b8a6a3b
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 04:30 UTC

[V1][Bugfix] Fix EngineCoreProc profile (#11185)

Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>

Tyler Michael Smith committed Dec 13, 2024 at 20:02 UTC 4b5b8a6a3bd94d9b0248b36b0eb4739d76fbb386
1 file changed +1 -1
vllm/v1/engine/core.py
+1 -1
@@ -321,7 +321,7 @@ class EngineCoreProc(EngineCore):
321 if isinstance(request, EngineCoreRequest):
322 self.add_request(request)
323 elif isinstance(request, EngineCoreProfile):
324 - self.model_executor.worker.profile(request.is_start)
324 + self.model_executor.profile(request.is_start)
325 else:
326 # TODO: make an EngineCoreAbort wrapper
327 assert isinstance(request, list)