A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 13:30 UTC
[Doc] V1 : Update LoRA status (#17133)
Signed-off-by: varun sundar rabindranath <vsundarr@redhat.com> Co-authored-by: varun sundar rabindranath <vsundarr@redhat.com>
Varun Sundar Rabindranath committed
Apr 24, 2025 at 23:17 UTC
7a0a9da72bad7c649d4c74b363040e0744e5d37d
1 file changed
+1
-6
docs/source/getting_started/v1_user_guide.md
+1
-6
@@ -44,8 +44,8 @@ This living user guide outlines a few known **important changes and limitations*
44
|-----------------|-----------------------------------------------------------------------------------|
45
| **Prefix Caching** | <nobr>🚀 Optimized</nobr> |
46
| **Chunked Prefill** | <nobr>🚀 Optimized</nobr> |
47
+| **LoRA** | <nobr>🚀 Optimized</nobr> |
48
| **Logprobs Calculation** | <nobr>🟢 Functional</nobr> |
48
-| **LoRA** | <nobr>🟢 Functional ([PR #13096](https://github.com/vllm-project/vllm/pull/13096))</nobr>|
49
| **Multimodal Models** | <nobr>🟢 Functional</nobr> |
50
| **FP8 KV Cache** | <nobr>🟢 Functional on Hopper devices ([PR #15191](https://github.com/vllm-project/vllm/pull/15191))</nobr>|
51
| **Spec Decode** | <nobr>🚧 WIP ([PR #13933](https://github.com/vllm-project/vllm/pull/13933))</nobr>|
@@ -121,11 +121,6 @@ Although we have re-implemented and partially optimized many features and models
121
These features are already supported in vLLM V1, but their optimization is still
122
in progress.
123
124
-- **LoRA**: LoRA is functionally working on vLLM V1 but its performance is
125
- inferior to that of V0. The team is actively working on improving its
126
- performance
127
-(e.g., see [PR #13096](https://github.com/vllm-project/vllm/pull/13096)).
128
-
124
- **Spec Decode**: Currently, only ngram-based spec decode is supported in V1. There
125
will be follow-up work to support other types of spec decode (e.g., see [PR #13933](https://github.com/vllm-project/vllm/pull/13933)). We will prioritize the support for Eagle, MTP compared to draft model based spec decode.
126