@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 446c7806b2
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 07:00 UTC

[Minor] Fix line-too-long (#10563)

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Woosuk Kwon committed Nov 21, 2024 at 19:40 UTC 446c7806b21d810b90604097487cc87393542aad
1 file changed +4 -4
vllm/entrypoints/llm.py
+4 -4
@@ -109,16 +109,16 @@ class LLM:
109 hf_overrides: If a dictionary, contains arguments to be forwarded to the
110 HuggingFace config. If a callable, it is called to update the
111 HuggingFace config.
112 - compilation_config: Either an integer or a dictionary. If it is an integer,
113 - it is used as the level of compilation optimization. If it is a dictionary,
114 - it can specify the full compilation configuration.
112 + compilation_config: Either an integer or a dictionary. If it is an
113 + integer, it is used as the level of compilation optimization. If it
114 + is a dictionary, it can specify the full compilation configuration.
115 **kwargs: Arguments for :class:`~vllm.EngineArgs`. (See
116 :ref:`engine_args`)
117
118 Note:
119 This class is intended to be used for offline inference. For online
120 serving, use the :class:`~vllm.AsyncLLMEngine` class instead.
121 - """ # noqa
121 + """
122
123 DEPRECATE_LEGACY: ClassVar[bool] = False
124 """A flag to toggle whether to deprecate the legacy generate/encode API."""