A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 1, 20:00 UTC
[Doc] Add Llama-3.2-3B-Instruct to batch-invariance tested models (#44435)
Signed-off-by: Daoyuan Li <94409450+DaoyuanLi2816@users.noreply.github.com>
Daoyuan Li committed
Jun 5, 2026 at 16:04 UTC
f6a708ab2bfd1278d4ca54ed150295efb3f16ed5
1 file changed
+2
-5
docs/features/batch_invariance.md
+2
-5
@@ -17,10 +17,7 @@ Batch invariance is crucial for several use cases:
17
18
## Hardware Requirements
19
20
-Batch invariance currently requires NVIDIA GPUs with compute capability 9.0 or higher:
21
-
22
-- **H-series**: H100, H200
23
-- **B-series**: B100, B200
20
+Batch invariance requires NVIDIA GPUs with compute capability 8.0 or higher.
21
22
## Enabling Batch Invariance
23
@@ -107,7 +104,7 @@ Batch invariance has been tested and verified on the following models:
104
- **Qwen3 (Dense)**: `Qwen/Qwen3-1.7B`, `Qwen/Qwen3-8B`, `Qwen/Qwen3-4B-AWQ`, `Qwen/Qwen3-8B-AWQ`
105
- **Qwen3 (MoE)**: `Qwen/Qwen3-30B-A3B`, `Qwen/Qwen3-Next-80B-A3B-Instruct`, `Qwen/Qwen3-30B-A3B-Thinking-2507-FP8`
106
- **Qwen2.5**: `Qwen/Qwen2.5-0.5B-Instruct`, `Qwen/Qwen2.5-1.5B-Instruct`, `Qwen/Qwen2.5-3B-Instruct`, `Qwen/Qwen2.5-7B-Instruct`, `Qwen/Qwen2.5-14B-Instruct`, `Qwen/Qwen2.5-32B-Instruct`
110
-- **Llama 3**: `meta-llama/Llama-3.1-8B-Instruct`, `meta-llama/Llama-3.2-1B-Instruct`
107
+- **Llama 3**: Llama3.1 and 3.2 series, `meta-llama/Llama-3.2-3B-Instruct` for example
108
- **GPT-OSS**: `openai/gpt-oss-20b`, `openai/gpt-oss-120b`
109
- **Mistral**: `mistralai/Mistral-7B-v0.3`
110