A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 2, 18:15 UTC
[Docs] Fix MLA prefill backend default docs (#43697)
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
Mohammad Miadh Angkad committed
May 27, 2026 at 18:13 UTC
158289e0fce93f633a7856cb49cca1bed107a4bb
2 files changed
+7
-5
docs/design/attention_backends.md
+3
-2
@@ -205,8 +205,9 @@ hardware and configuration.
205
| `FLASHINFER` | FlashInfer CUTLASS backend | fp16, bf16 | 10.x | DeepSeek R1 dims only |
206
| `TOKENSPEED_MLA` | | fp16, bf16 | 10.x | DeepSeek R1 dims only |
207
208
-> **‡** TRT-LLM Ragged is the default on Blackwell (SM100).
209
-> On other GPUs, FlashAttention is used as the default.
208
+> **‡** Automatic selection tries FlashAttention first. On Blackwell
209
+> (SM100), the fallback order is TRT-LLM Ragged, FlashInfer, then
210
+> TokenSpeed MLA. On other GPUs, only FlashAttention is considered.
211
212
### Decode Backends
213
tools/pre_commit/generate_attention_backend_docs.py
+4
-3
@@ -508,7 +508,7 @@ def parse_mla_prefill_backends() -> list[dict[str, Any]]:
508
metadata = backend_metadata.get(backend_name, {})
509
display_name = backend_info.get("name", backend_name)
510
511
- # Add marker for default Blackwell backend
511
+ # Add marker for the highest-priority automatic backend.
512
marker = ""
513
if backend_name == priority_order[0] and priorities.get("blackwell"):
514
marker = "‡"
@@ -1595,8 +1595,9 @@ def generate_mla_section(
1595
lines.extend(
1596
[
1597
"",
1598
- "> **‡** TRT-LLM Ragged is the default on Blackwell (SM100).",
1599
- "> On other GPUs, FlashAttention is used as the default.",
1598
+ "> **‡** Automatic selection tries FlashAttention first. On Blackwell",
1599
+ "> (SM100), the fallback order is TRT-LLM Ragged, FlashInfer, then",
1600
+ "> TokenSpeed MLA. On other GPUs, only FlashAttention is considered.",
1601
"",
1602
"### Decode Backends",
1603
"",