A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 03:00 UTC
[Misc] Fix typo in scheduler.py (#4022)
Zhuohan Li committed
Apr 12, 2024 at 13:56 UTC
d4ec9ffb9574988132d927fd1615180522877262
1 file changed
+1
-1
vllm/core/scheduler.py
+1
-1
@@ -674,7 +674,7 @@ class Scheduler:
674
def _schedule_default(self) -> SchedulerOutputs:
675
"""Schedule queued requests.
676
677
- The current policy is designed to opimimize the throughput. First,
677
+ The current policy is designed to optimize the throughput. First,
678
it batches as many prefill requests as possible. And it schedules
679
decodes. If there's a pressure on GPU memory, decode requests can
680
be swapped or preempted.