@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / a30a605d21
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 07:00 UTC

[Doc] Add encoder-based models to Supported Models page (#10616)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Cyrus Leung committed Nov 25, 2024 at 14:34 UTC a30a605d214e051c31057f8c0cb948c841a2f743
1 file changed +45
docs/source/models/supported_models.rst
+45
@@ -325,6 +325,11 @@ Text Embedding
325 - Example HF Models
326 - :ref:`LoRA <lora>`
327 - :ref:`PP <distributed_serving>`
328 + * - :code:`BertModel`
329 + - BERT-based
330 + - :code:`BAAI/bge-base-en-v1.5`, etc.
331 + -
332 + -
333 * - :code:`Gemma2Model`
334 - Gemma2-based
335 - :code:`BAAI/bge-multilingual-gemma2`, etc.
@@ -340,6 +345,16 @@ Text Embedding
345 - :code:`ssmits/Qwen2-7B-Instruct-embed-base`, :code:`Alibaba-NLP/gte-Qwen2-1.5B-instruct`, etc.
346 - ✅︎
347 - ✅︎
348 + * - :code:`RobertaModel`, :code:`RobertaForMaskedLM`
349 + - RoBERTa-based
350 + - :code:`sentence-transformers/all-roberta-large-v1`, :code:`sentence-transformers/all-roberta-large-v1`, etc.
351 + -
352 + -
353 + * - :code:`XLMRobertaModel`
354 + - XLM-RoBERTa-based
355 + - :code:`intfloat/multilingual-e5-large`, etc.
356 + -
357 + -
358
359 .. important::
360 Some model architectures support both generation and embedding tasks.
@@ -390,6 +405,36 @@ Classification
405 .. note::
406 As an interim measure, these models are supported in both offline and online inference via Embeddings API.
407
408 +Sentence Pair Scoring
409 +---------------------
410 +
411 +.. list-table::
412 + :widths: 25 25 50 5 5
413 + :header-rows: 1
414 +
415 + * - Architecture
416 + - Models
417 + - Example HF Models
418 + - :ref:`LoRA <lora>`
419 + - :ref:`PP <distributed_serving>`
420 + * - :code:`BertForSequenceClassification`
421 + - BERT-based
422 + - :code:`cross-encoder/ms-marco-MiniLM-L-6-v2`, etc.
423 + -
424 + -
425 + * - :code:`RobertaForSequenceClassification`
426 + - RoBERTa-based
427 + - :code:`cross-encoder/quora-roberta-base`, etc.
428 + -
429 + -
430 + * - :code:`XLMRobertaForSequenceClassification`
431 + - XLM-RoBERTa-based
432 + - :code:`BAAI/bge-reranker-v2-m3`, etc.
433 + -
434 + -
435 +
436 +.. note::
437 + These models are supported in both offline and online inference via Score API.
438
439 Multimodal Language Models
440 ^^^^^^^^^^^^^^^^^^^^^^^^^^