@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / afed90a034
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 23:15 UTC

[Frontend][Bugfix] Fix preemption_mode -> preemption-mode for CLI arg in arg_utils.py (#5688)

Michael Goin committed Jun 19, 2024 at 14:41 UTC afed90a0344b1b0ce6aae46efc630adb489ec769
1 file changed +1 -1
vllm/engine/arg_utils.py
+1 -1
@@ -577,7 +577,7 @@ class EngineArgs:
577 'This should be a JSON string that will be '
578 'parsed into a dictionary.')
579 parser.add_argument(
580 - '--preemption_mode',
580 + '--preemption-mode',
581 type=str,
582 default=None,
583 help='If \'recompute\', the engine performs preemption by block '