A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 17:15 UTC
[Doc] Update V1 user guide for multi-modality (#15460)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Cyrus Leung committed
Mar 25, 2025 at 19:01 UTC
3f04a7fbf21e1b0a899c721ebc6853c3cf030db7
1 file changed
+3
docs/source/getting_started/v1_user_guide.md
+3
@@ -129,6 +129,9 @@ in progress.
129
- **Spec Decode**: Currently, only ngram-based spec decode is supported in V1. There
130
will be follow-up work to support other types of spec decode (e.g., see [PR #13933](https://github.com/vllm-project/vllm/pull/13933)). We will prioritize the support for Eagle, MTP compared to draft model based spec decode.
131
132
+- **Multimodal Models**: V1 is almost fully compatible with V0 except that interleaved modality input is not supported yet.
133
+ See [here](https://github.com/orgs/vllm-project/projects/8) for the status of upcoming features and optimizations.
134
+
135
#### Features to Be Supported
136
137
- **FP8 KV Cache**: While vLLM V1 introduces new FP8 kernels for model weight quantization, support for an FP8 key–value cache is not yet available. Users must continue using FP16 (or other supported precisions) for the KV cache.