A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 09:30 UTC
[Doc] Update Qwen documentation (#9869)
Jee Jee Li committed
Oct 31, 2024 at 16:54 UTC
5608e611c2116cc17c6808b2ae1ecb4a3e263493
2 files changed
+6
-3
docs/source/models/supported_models.rst
+5
-2
@@ -277,7 +277,7 @@ Text Generation
277
* - :code:`QWenLMHeadModel`
278
- Qwen
279
- :code:`Qwen/Qwen-7B`, :code:`Qwen/Qwen-7B-Chat`, etc.
280
- -
280
+ - ✅︎
281
- ✅︎
282
* - :code:`Qwen2ForCausalLM`
283
- Qwen2
@@ -516,7 +516,7 @@ Text Generation
516
- Qwen-VL
517
- T + I\ :sup:`E+`
518
- :code:`Qwen/Qwen-VL`, :code:`Qwen/Qwen-VL-Chat`, etc.
519
- -
519
+ - ✅︎
520
- ✅︎
521
* - :code:`Qwen2AudioForConditionalGeneration`
522
- Qwen2-Audio
@@ -540,6 +540,9 @@ Text Generation
540
| :sup:`E` Pre-computed embeddings can be inputted for this modality.
541
| :sup:`+` Multiple items can be inputted per text prompt for this modality.
542
543
+.. note::
544
+ vLLM currently only supports adding LoRA to the language backbone of multimodal models.
545
+
546
.. note::
547
For :code:`openbmb/MiniCPM-V-2`, the official repo doesn't work yet, so we need to use a fork (:code:`HwwwH/MiniCPM-V-2`) for now.
548
For more details, please see: https://github.com/vllm-project/vllm/pull/4087#issuecomment-2250397630
vllm/model_executor/models/qwen.py
+1
-1
@@ -1048,7 +1048,7 @@ class QWenVL(QWenBaseModel):
1048
@MULTIMODAL_REGISTRY.register_max_image_tokens(MAX_QWEN_IMG_TOKENS)
1049
@INPUT_REGISTRY.register_dummy_data(dummy_data_for_qwen)
1050
@INPUT_REGISTRY.register_input_processor(input_processor_for_qwen)
1051
-class QWenLMHeadModel(QWenBaseModel):
1051
+class QWenLMHeadModel(QWenBaseModel, SupportsLoRA):
1052
"""
1053
QWenLMHeadModel is not only applicable to LLM but also to VL, which is not
1054
conducive to the current integration logic of LoRA in vLLM. Therefore, it