@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 5608e611c2
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 09:30 UTC

[Doc] Update Qwen documentation (#9869)

Jee Jee Li committed Oct 31, 2024 at 16:54 UTC 5608e611c2116cc17c6808b2ae1ecb4a3e263493
2 files changed +6 -3
docs/source/models/supported_models.rst
+5 -2
@@ -277,7 +277,7 @@ Text Generation
277 * - :code:`QWenLMHeadModel`
278 - Qwen
279 - :code:`Qwen/Qwen-7B`, :code:`Qwen/Qwen-7B-Chat`, etc.
280 - -
280 + - ✅︎
281 - ✅︎
282 * - :code:`Qwen2ForCausalLM`
283 - Qwen2
@@ -516,7 +516,7 @@ Text Generation
516 - Qwen-VL
517 - T + I\ :sup:`E+`
518 - :code:`Qwen/Qwen-VL`, :code:`Qwen/Qwen-VL-Chat`, etc.
519 - -
519 + - ✅︎
520 - ✅︎
521 * - :code:`Qwen2AudioForConditionalGeneration`
522 - Qwen2-Audio
@@ -540,6 +540,9 @@ Text Generation
540 | :sup:`E` Pre-computed embeddings can be inputted for this modality.
541 | :sup:`+` Multiple items can be inputted per text prompt for this modality.
542
543 +.. note::
544 + vLLM currently only supports adding LoRA to the language backbone of multimodal models.
545 +
546 .. note::
547 For :code:`openbmb/MiniCPM-V-2`, the official repo doesn't work yet, so we need to use a fork (:code:`HwwwH/MiniCPM-V-2`) for now.
548 For more details, please see: https://github.com/vllm-project/vllm/pull/4087#issuecomment-2250397630
vllm/model_executor/models/qwen.py
+1 -1
@@ -1048,7 +1048,7 @@ class QWenVL(QWenBaseModel):
1048 @MULTIMODAL_REGISTRY.register_max_image_tokens(MAX_QWEN_IMG_TOKENS)
1049 @INPUT_REGISTRY.register_dummy_data(dummy_data_for_qwen)
1050 @INPUT_REGISTRY.register_input_processor(input_processor_for_qwen)
1051 -class QWenLMHeadModel(QWenBaseModel):
1051 +class QWenLMHeadModel(QWenBaseModel, SupportsLoRA):
1052 """
1053 QWenLMHeadModel is not only applicable to LLM but also to VL, which is not
1054 conducive to the current integration logic of LoRA in vLLM. Therefore, it