A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 6, 13:15 UTC
[Model] Enable LoRA support for tower and connector in H2OVL (#31696)
Signed-off-by: shwetha-s-poojary <shwetha.s-poojary@ibm.com>
Shwetha Poojary committed
Mar 18, 2026 at 18:56 UTC
cef1f302d27b0152761509e5297b831db41a146a
2 files changed
+15
-1
docs/models/supported_models.md
+1
-1
@@ -707,7 +707,7 @@ These models primarily accept the [`LLM.generate`](./generative_models.md#llmgen
707
| `GraniteSpeechForConditionalGeneration` | Granite Speech | T + A | `ibm-granite/granite-speech-3.3-8b` | ✅︎ | ✅︎ |
708
| `HCXVisionForCausalLM` | HyperCLOVAX-SEED-Vision-Instruct-3B | T + I<sup>+</sup> + V<sup>+</sup> | `naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B` | | |
709
| `HCXVisionV2ForCausalLM` | HyperCLOVAX-SEED-Think-32B | T + I<sup>+</sup> + V<sup>+</sup> | `naver-hyperclovax/HyperCLOVAX-SEED-Think-32B` | | |
710
-| `H2OVLChatModel` | H2OVL | T + I<sup>E+</sup> | `h2oai/h2ovl-mississippi-800m`, `h2oai/h2ovl-mississippi-2b`, etc. | | ✅︎ |
710
+| `H2OVLChatModel` | H2OVL | T + I<sup>E+</sup> | `h2oai/h2ovl-mississippi-800m`, `h2oai/h2ovl-mississippi-2b`, etc. | ✅︎ | ✅︎ |
711
| `HunYuanVLForConditionalGeneration` | HunyuanOCR | T + I<sup>E+</sup> | `tencent/HunyuanOCR`, etc. | ✅︎ | ✅︎ |
712
| `Idefics3ForConditionalGeneration` | Idefics3 | T + I | `HuggingFaceM4/Idefics3-8B-Llama3`, etc. | ✅︎ | |
713
| `IsaacForConditionalGeneration` | Isaac | T + I<sup>+</sup> | `PerceptronAI/Isaac-0.1` | ✅︎ | ✅︎ |
vllm/model_executor/models/h2ovl.py
+14
@@ -163,3 +163,17 @@ class H2OVLChatModel(InternVLChatModel):
163
else:
164
msg = "Monolith mode is not applicable to H2OVL"
165
raise NotImplementedError(msg)
166
+
167
+ def get_num_mm_encoder_tokens(self, num_image_tokens: int) -> int:
168
+ if num_image_tokens <= 0 or self.num_image_token <= 0:
169
+ return 0
170
+
171
+ num_patches = num_image_tokens // self.num_image_token
172
+ return num_patches * (self.patch_tokens + 1)
173
+
174
+ def get_num_mm_connector_tokens(self, num_vision_tokens: int) -> int:
175
+ if num_vision_tokens <= 0 or self.num_image_token <= 0:
176
+ return 0
177
+
178
+ num_patches = num_vision_tokens // (self.patch_tokens + 1)
179
+ return num_patches * self.num_image_token