A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 07:15 UTC
[Doc] Update V1 Guide for embedding models (#19141)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Cyrus Leung committed
Jun 4, 2025 at 22:57 UTC
8f4ffbd373cb19e8f8dcfa6dec1dbbe98fbeae96
1 file changed
+3
-3
docs/usage/v1_guide.md
+3
-3
@@ -55,7 +55,7 @@ This living user guide outlines a few known **important changes and limitations*
55
| **Spec Decode** | <nobr>🚧 WIP ([PR #13933](https://github.com/vllm-project/vllm/pull/13933))</nobr>|
56
| **Prompt Logprobs with Prefix Caching** | <nobr>🟡 Planned ([RFC #13414](https://github.com/vllm-project/vllm/issues/13414))</nobr>|
57
| **Structured Output Alternative Backends** | <nobr>🟡 Planned</nobr> |
58
-| **Embedding Models** | <nobr>🚧 WIP ([PR #18015](https://github.com/vllm-project/vllm/pull/18015))</nobr> |
58
+| **Embedding Models** | <nobr>🚧 WIP ([PR #16188](https://github.com/vllm-project/vllm/pull/16188))</nobr> |
59
| **Mamba Models** | <nobr>🟡 Planned</nobr> |
60
| **Encoder-Decoder Models** | <nobr>🟠 Delayed</nobr> |
61
| **Request-level Structured Output Backend** | <nobr>🔴 Deprecated</nobr> |
@@ -145,9 +145,9 @@ vLLM V1 currently excludes model architectures with the `SupportsV0Only` protoco
145
and the majority fall into the following categories. V1 support for these models will be added eventually.
146
147
**Embedding Models**
148
-Initially, we will create a [separate model runner](https://github.com/vllm-project/vllm/pull/18015) to provide V1 support without conflicting with other ongoing work.
148
+The initial support will be provided by [PR #16188](https://github.com/vllm-project/vllm/pull/16188).
149
150
-Later, we will consider using [hidden states processor](https://github.com/vllm-project/vllm/issues/12249), which is based on [global logits processor](https://github.com/vllm-project/vllm/pull/13360) to enable simultaneous generation and embedding using the same engine instance in V1. [PR #16188](https://github.com/vllm-project/vllm/pull/16188) is the first step towards enabling this.
150
+Later, we will consider using [hidden states processor](https://github.com/vllm-project/vllm/issues/12249), which is based on [global logits processor](https://github.com/vllm-project/vllm/pull/13360) to enable simultaneous generation and embedding using the same engine instance in V1.
151
152
**Mamba Models**
153
Models using selective state-space mechanisms (instead of standard transformer attention)