A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 07:00 UTC
[Minor] Fix line-too-long (#10563)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Woosuk Kwon committed
Nov 21, 2024 at 19:40 UTC
446c7806b21d810b90604097487cc87393542aad
1 file changed
+4
-4
vllm/entrypoints/llm.py
+4
-4
@@ -109,16 +109,16 @@ class LLM:
109
hf_overrides: If a dictionary, contains arguments to be forwarded to the
110
HuggingFace config. If a callable, it is called to update the
111
HuggingFace config.
112
- compilation_config: Either an integer or a dictionary. If it is an integer,
113
- it is used as the level of compilation optimization. If it is a dictionary,
114
- it can specify the full compilation configuration.
112
+ compilation_config: Either an integer or a dictionary. If it is an
113
+ integer, it is used as the level of compilation optimization. If it
114
+ is a dictionary, it can specify the full compilation configuration.
115
**kwargs: Arguments for :class:`~vllm.EngineArgs`. (See
116
:ref:`engine_args`)
117
118
Note:
119
This class is intended to be used for offline inference. For online
120
serving, use the :class:`~vllm.AsyncLLMEngine` class instead.
121
- """ # noqa
121
+ """
122
123
DEPRECATE_LEGACY: ClassVar[bool] = False
124
"""A flag to toggle whether to deprecate the legacy generate/encode API."""