A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
Officially support SmolLM3 using the Transformers backend (#22665)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Harry Mellor committed
Aug 12, 2025 at 13:38 UTC
80bb1e8afe950342e93b7262e7bf25eb6d29b287
3 files changed
+10
docs/models/supported_models.md
+6
@@ -409,6 +409,12 @@ th {
409
| `MiniMaxText01ForCausalLM` | MiniMax-Text | `MiniMaxAI/MiniMax-Text-01`, etc. | | | ✅︎ |
410
| `Zamba2ForCausalLM` | Zamba2 | `Zyphra/Zamba2-7B-instruct`, `Zyphra/Zamba2-2.7B-instruct`, `Zyphra/Zamba2-1.2B-instruct`, etc. | | | ✅︎ |
411
412
+Some models are supported only via the [Transformers backend](#transformers). The purpose of the table below is to acknowledge models which we officially support in this way. The logs will say that the Transformers backend is being used, and you will see no warning that this is fallback behaviour. This means that, if you have issues with any of the models listed below, please [make an issue](https://github.com/vllm-project/vllm/issues/new/choose) and we'll do our best to fix it!
413
+
414
+| Architecture | Models | Example HF Models | [LoRA](../features/lora.md) | [PP](../serving/parallelism_scaling.md) | [V1](gh-issue:8779) |
415
+|--------------|--------|-------------------|----------------------|---------------------------|---------------------|
416
+| `SmolLM3ForCausalLM` | SmolLM3 | `HuggingFaceTB/SmolLM3-3B` | ✅︎ | ✅︎ | ✅︎ |
417
+
418
!!! note
419
Currently, the ROCm version of vLLM supports Mistral and Mixtral only for context lengths up to 4096.
420
tests/models/registry.py
+1
@@ -291,6 +291,7 @@ _TEXT_GENERATION_EXAMPLE_MODELS = {
291
"Qwen3ForCausalLM": _HfExamplesInfo("Qwen/Qwen3-8B"),
292
"Qwen3MoeForCausalLM": _HfExamplesInfo("Qwen/Qwen3-30B-A3B"),
293
"RWForCausalLM": _HfExamplesInfo("tiiuae/falcon-40b"),
294
+ "SmolLM3ForCausalLM": _HfExamplesInfo("HuggingFaceTB/SmolLM3-3B"),
295
"StableLMEpochForCausalLM": _HfExamplesInfo("stabilityai/stablelm-zephyr-3b"), # noqa: E501
296
"StableLmForCausalLM": _HfExamplesInfo("stabilityai/stablelm-3b-4e1t"),
297
"Starcoder2ForCausalLM": _HfExamplesInfo("bigcode/starcoder2-3b"),
vllm/model_executor/models/registry.py
+3
@@ -271,6 +271,9 @@ _SPECULATIVE_DECODING_MODELS = {
271
}
272
273
_TRANSFORMERS_SUPPORTED_MODELS = {
274
+ # Text generation models
275
+ "SmolLM3ForCausalLM": ("transformers", "TransformersForCausalLM"),
276
+ # Multimodal models
277
"Emu3ForConditionalGeneration": ("transformers", "TransformersForMultimodalLM"), # noqa: E501
278
}
279