[Model][LoRA] Enable LoRA support for ModernBertModel (#57148)
Signed-off-by: linitra24 <renshuang.zhou@daocloud.io>
linitra24 committed
Sep 16, 2026 at 22:16 UTC
ca542ee154e91c9a7757fb0b8f9d90ee578fb613
2 files changed
+8
-3
docs/models/pooling_models/embed.md
+1
-1
@@ -47,7 +47,7 @@ You can compute pairwise similarity scores to build a similarity matrix using th
47
| `JinaEmbeddingsV5Model`<sup>C</sup> | Qwen3-decoder or EuroBERT-encoder backbone with task-specific LoRA adapters | `jinaai/jina-embeddings-v5-text-small`, `jinaai/jina-embeddings-v5-text-nano` (see note) | ✅︎ | ✅︎ |
48
| `LlamaBidirectionalModel`<sup>C</sup> | Llama-based with bidirectional attention | `nvidia/llama-nemotron-embed-1b-v2`, etc. | ✅︎ | ✅︎ |
49
| `LlamaModel`<sup>C</sup>, `LlamaForCausalLM`<sup>C</sup>, `MistralModel`<sup>C</sup>, etc. | Llama-based | `intfloat/e5-mistral-7b-instruct`, etc. | ✅︎ | ✅︎ |
50
-| `ModernBertModel` | ModernBERT-based | `Alibaba-NLP/gte-modernbert-base`, etc. | | |
50
+| `ModernBertModel` | ModernBERT-based | `Alibaba-NLP/gte-modernbert-base`, etc. | ✅︎ | |
51
| `NomicBertModel` | Nomic BERT | `nomic-ai/nomic-embed-text-v1`, `nomic-ai/nomic-embed-text-v2-moe`, `Snowflake/snowflake-arctic-embed-m-long`, etc. | | |
52
| `Qwen2Model`<sup>C</sup>, `Qwen2ForCausalLM`<sup>C</sup> | Qwen2-based | `ssmits/Qwen2-7B-Instruct-embed-base` (see note), `Alibaba-NLP/gte-Qwen2-7B-instruct` (see note), etc. | ✅︎ | ✅︎ |
53
| `Qwen3Model`<sup>C</sup>, `Qwen3ForCausalLM`<sup>C</sup> | Qwen3-based | `Qwen/Qwen3-Embedding-0.6B`, etc. | ✅︎ | ✅︎ |
vllm/model_executor/models/modernbert.py
+7
-2
@@ -32,7 +32,7 @@ from vllm.model_executor.layers.vocab_parallel_embedding import VocabParallelEmb
32
from vllm.model_executor.model_loader.weight_utils import default_weight_loader
33
from vllm.sequence import IntermediateTensors
34
35
-from .interfaces import SupportsCrossEncoding
35
+from .interfaces import SupportsCrossEncoding, SupportsLoRA
36
from .interfaces_base import attn_type, default_pooling_type
37
from .utils import AutoWeightsLoader, WeightsMapper, maybe_prefix
38
@@ -262,7 +262,7 @@ class ModernBertEncoderLayer(nn.Module):
262
263
@support_torch_compile
264
@default_pooling_type(seq_pooling_type="CLS")
265
-class ModernBertModel(nn.Module):
265
+class ModernBertModel(nn.Module, SupportsLoRA):
266
hf_to_vllm_mapper = WeightsMapper(
267
orig_to_new_prefix={
268
"model.layers.": "encoder_layer.layers.",
@@ -271,6 +271,11 @@ class ModernBertModel(nn.Module):
271
}
272
)
273
274
+ packed_modules_mapping = {
275
+ "Wqkv": ["Wqkv"],
276
+ "Wi": ["Wi"],
277
+ }
278
+
279
def __init__(
280
self,
281
vllm_config: VllmConfig,