A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 07:00 UTC
[Doc] Add encoder-based models to Supported Models page (#10616)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Cyrus Leung committed
Nov 25, 2024 at 14:34 UTC
a30a605d214e051c31057f8c0cb948c841a2f743
1 file changed
+45
docs/source/models/supported_models.rst
+45
@@ -325,6 +325,11 @@ Text Embedding
325
- Example HF Models
326
- :ref:`LoRA <lora>`
327
- :ref:`PP <distributed_serving>`
328
+ * - :code:`BertModel`
329
+ - BERT-based
330
+ - :code:`BAAI/bge-base-en-v1.5`, etc.
331
+ -
332
+ -
333
* - :code:`Gemma2Model`
334
- Gemma2-based
335
- :code:`BAAI/bge-multilingual-gemma2`, etc.
@@ -340,6 +345,16 @@ Text Embedding
345
- :code:`ssmits/Qwen2-7B-Instruct-embed-base`, :code:`Alibaba-NLP/gte-Qwen2-1.5B-instruct`, etc.
346
- ✅︎
347
- ✅︎
348
+ * - :code:`RobertaModel`, :code:`RobertaForMaskedLM`
349
+ - RoBERTa-based
350
+ - :code:`sentence-transformers/all-roberta-large-v1`, :code:`sentence-transformers/all-roberta-large-v1`, etc.
351
+ -
352
+ -
353
+ * - :code:`XLMRobertaModel`
354
+ - XLM-RoBERTa-based
355
+ - :code:`intfloat/multilingual-e5-large`, etc.
356
+ -
357
+ -
358
359
.. important::
360
Some model architectures support both generation and embedding tasks.
@@ -390,6 +405,36 @@ Classification
405
.. note::
406
As an interim measure, these models are supported in both offline and online inference via Embeddings API.
407
408
+Sentence Pair Scoring
409
+---------------------
410
+
411
+.. list-table::
412
+ :widths: 25 25 50 5 5
413
+ :header-rows: 1
414
+
415
+ * - Architecture
416
+ - Models
417
+ - Example HF Models
418
+ - :ref:`LoRA <lora>`
419
+ - :ref:`PP <distributed_serving>`
420
+ * - :code:`BertForSequenceClassification`
421
+ - BERT-based
422
+ - :code:`cross-encoder/ms-marco-MiniLM-L-6-v2`, etc.
423
+ -
424
+ -
425
+ * - :code:`RobertaForSequenceClassification`
426
+ - RoBERTa-based
427
+ - :code:`cross-encoder/quora-roberta-base`, etc.
428
+ -
429
+ -
430
+ * - :code:`XLMRobertaForSequenceClassification`
431
+ - XLM-RoBERTa-based
432
+ - :code:`BAAI/bge-reranker-v2-m3`, etc.
433
+ -
434
+ -
435
+
436
+.. note::
437
+ These models are supported in both offline and online inference via Score API.
438
439
Multimodal Language Models
440
^^^^^^^^^^^^^^^^^^^^^^^^^^