A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 01:00 UTC
[Docs] [V1] Update docs to remove enforce_eager limitation for hybrid models. (#21233)
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Thomas Parnell committed
Jul 20, 2025 at 01:09 UTC
2b504eb77031cfc947a9990ead42c8bc8baa98c5
1 file changed
+2
-3
docs/usage/v1_guide.md
+2
-3
@@ -107,12 +107,11 @@ to enable simultaneous generation and embedding using the same engine instance i
107
Models using selective state-space mechanisms instead of standard transformer attention are partially supported.
108
Models that use Mamba-2 layers (e.g., `Mamba2ForCausalLM`) are supported, but models that use older Mamba-1 layers
109
(e.g., `MambaForCausalLM`, `JambaForCausalLM`) are not yet supported. Please note that these models currently require
110
-enforcing eager mode and disabling prefix caching in V1.
110
+disabling prefix caching in V1.
111
112
Models that combine Mamba-2 layers with standard attention layers are also supported (e.g., `BambaForCausalLM`,
113
`Zamba2ForCausalLM`, `NemotronHForCausalLM`, `FalconH1ForCausalLM` and `GraniteMoeHybridForCausalLM`). Please note that
114
-these models currently require enforcing eager mode, disabling prefix caching, and using the FlashInfer attention
115
-backend in V1.
114
+these models currently require disabling prefix caching and using the FlashInfer attention backend in V1.
115
116
#### Encoder-Decoder Models
117