@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / f22619fe96
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 10:45 UTC

[Misc] Remove user-facing error for removed VLM args (#9104)

Cyrus Leung committed Oct 6, 2024 at 16:33 UTC f22619fe96c842ee2406638678d2b60009d8ff14
2 files changed +1 -13
docs/source/models/vlm.rst
-4
@@ -23,10 +23,6 @@ The :class:`~vllm.LLM` class can be instantiated in much the same way as languag
23
24 llm = LLM(model="llava-hf/llava-1.5-7b-hf")
25
26 -.. note::
27 - We have removed all vision language related CLI args in the ``0.5.1`` release. **This is a breaking change**, so please update your code to follow
28 - the above snippet. Specifically, ``image_feature_size`` can no longer be specified as we now calculate that internally for each model.
29 -
26 To pass an image to the model, note the following in :class:`vllm.inputs.PromptType`:
27
28 * ``prompt``: The prompt should follow the format that is documented on HuggingFace.
vllm/entrypoints/llm.py
+1 -9
@@ -180,15 +180,7 @@ class LLM:
180
181 if "disable_log_stats" not in kwargs:
182 kwargs["disable_log_stats"] = True
183 - removed_vision_keys = (
184 - "image_token_id",
185 - "image_feature_size",
186 - "image_input_shape",
187 - "image_input_type",
188 - )
189 - if any(k in kwargs for k in removed_vision_keys):
190 - raise TypeError(
191 - "There is no need to pass vision-related arguments anymore.")
183 +
184 engine_args = EngineArgs(
185 model=model,
186 tokenizer=tokenizer,