@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / bbe4466fd9
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 05:30 UTC

[Minor] Fix typo (#2166)

Co-authored-by: John-Saxon <zhang.xiangxuan@oushu.com>

JohnSaxon committed Dec 18, 2023 at 15:28 UTC bbe4466fd98a27f918bd498c9e64a736202a133e
1 file changed +1 -1
vllm/engine/arg_utils.py
+1 -1
@@ -159,7 +159,7 @@ class EngineArgs:
159 parser.add_argument('--gpu-memory-utilization',
160 type=float,
161 default=EngineArgs.gpu_memory_utilization,
162 - help='the percentage of GPU memory to be used for'
162 + help='the percentage of GPU memory to be used for '
163 'the model executor')
164 parser.add_argument('--max-num-batched-tokens',
165 type=int,