@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / d4ec9ffb95
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 03:00 UTC

[Misc] Fix typo in scheduler.py (#4022)

Zhuohan Li committed Apr 12, 2024 at 13:56 UTC d4ec9ffb9574988132d927fd1615180522877262
1 file changed +1 -1
vllm/core/scheduler.py
+1 -1
@@ -674,7 +674,7 @@ class Scheduler:
674 def _schedule_default(self) -> SchedulerOutputs:
675 """Schedule queued requests.
676
677 - The current policy is designed to opimimize the throughput. First,
677 + The current policy is designed to optimize the throughput. First,
678 it batches as many prefill requests as possible. And it schedules
679 decodes. If there's a pressure on GPU memory, decode requests can
680 be swapped or preempted.