A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[Docs] [V1] [Hybrid] Update docs to remove FlashInfer constraint for hybrid models (#23665)
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Thomas Parnell committed
Aug 26, 2025 at 20:33 UTC
227e231b55901be4e050d5a8f033e90f45cfba85
1 file changed
+2
-3
docs/usage/v1_guide.md
+2
-3
@@ -111,11 +111,10 @@ Models that use Mamba-2 and Mamba-1 layers (e.g., `Mamba2ForCausalLM`, `MambaFor
111
112
Models that combine Mamba-2 and Mamba-1 layers with standard attention layers are also supported (e.g., `BambaForCausalLM`,
113
`Zamba2ForCausalLM`, `NemotronHForCausalLM`, `FalconH1ForCausalLM` and `GraniteMoeHybridForCausalLM`, `JambaForCausalLM`). Please note that
114
-these models currently require disabling prefix caching and using the FlashInfer attention backend in V1.
114
+these models currently require disabling prefix caching in V1.
115
116
Hybrid models with mechanisms different to Mamba are also supported (e.g, `MiniMaxText01ForCausalLM`, `MiniMaxM1ForCausalLM`).
117
-Please note that these models currently require disabling prefix caching, enforcing eager mode, and using the FlashInfer
118
-attention backend in V1.
117
+Please note that these models currently require disabling prefix caching and enforcing eager mode in V1.
118
119
#### Encoder-Decoder Models
120