A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 11:00 UTC
Make right sidebar more readable in "Supported Models" (#17723)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Harry Mellor committed
May 6, 2025 at 17:48 UTC
6115b115826040ad1f49b69a8b4fdd59f0df5113
1 file changed
+24
-8
docs/source/models/supported_models.md
+24
-8
@@ -239,7 +239,9 @@ print(output)
239
240
See [this page](#generative-models) for more information on how to use generative models.
241
242
-#### Text Generation (`--task generate`)
242
+#### Text Generation
243
+
244
+Specified using `--task generate`.
245
246
:::{list-table}
247
:widths: 25 25 50 5 5
@@ -605,7 +607,9 @@ Since some model architectures support both generative and pooling tasks,
607
you should explicitly specify the task type to ensure that the model is used in pooling mode instead of generative mode.
608
:::
609
608
-#### Text Embedding (`--task embed`)
610
+#### Text Embedding
611
+
612
+Specified using `--task embed`.
613
614
:::{list-table}
615
:widths: 25 25 50 5 5
@@ -670,7 +674,9 @@ If your model is not in the above list, we will try to automatically convert the
674
{func}`~vllm.model_executor.models.adapters.as_embedding_model`. By default, the embeddings
675
of the whole prompt are extracted from the normalized hidden state corresponding to the last token.
676
673
-#### Reward Modeling (`--task reward`)
677
+#### Reward Modeling
678
+
679
+Specified using `--task reward`.
680
681
:::{list-table}
682
:widths: 25 25 50 5 5
@@ -711,7 +717,9 @@ For process-supervised reward models such as `peiyi9979/math-shepherd-mistral-7b
717
e.g.: `--override-pooler-config '{"pooling_type": "STEP", "step_tag_id": 123, "returned_token_ids": [456, 789]}'`.
718
:::
719
714
-#### Classification (`--task classify`)
720
+#### Classification
721
+
722
+Specified using `--task classify`.
723
724
:::{list-table}
725
:widths: 25 25 50 5 5
@@ -737,7 +745,9 @@ e.g.: `--override-pooler-config '{"pooling_type": "STEP", "step_tag_id": 123, "r
745
If your model is not in the above list, we will try to automatically convert the model using
746
{func}`~vllm.model_executor.models.adapters.as_classification_model`. By default, the class probabilities are extracted from the softmaxed hidden state corresponding to the last token.
747
740
-#### Sentence Pair Scoring (`--task score`)
748
+#### Sentence Pair Scoring
749
+
750
+Specified using `--task score`.
751
752
:::{list-table}
753
:widths: 25 25 50 5 5
@@ -824,7 +834,9 @@ vLLM currently only supports adding LoRA to the language backbone of multimodal
834
835
See [this page](#generative-models) for more information on how to use generative models.
836
827
-#### Text Generation (`--task generate`)
837
+#### Text Generation
838
+
839
+Specified using `--task generate`.
840
841
:::{list-table}
842
:widths: 25 25 15 20 5 5 5
@@ -1200,7 +1212,9 @@ Since some model architectures support both generative and pooling tasks,
1212
you should explicitly specify the task type to ensure that the model is used in pooling mode instead of generative mode.
1213
:::
1214
1203
-#### Text Embedding (`--task embed`)
1215
+#### Text Embedding
1216
+
1217
+Specified using `--task embed`.
1218
1219
Any text generation model can be converted into an embedding model by passing `--task embed`.
1220
@@ -1240,7 +1254,9 @@ The following table lists those that are tested in vLLM.
1254
* ✅︎
1255
:::
1256
1243
-#### Transcription (`--task transcription`)
1257
+#### Transcription
1258
+
1259
+Specified using `--task transcription`.
1260
1261
Speech2Text models trained specifically for Automatic Speech Recognition.
1262