[XPU] update key supported models (#53494)
Signed-off-by: Yan Ma <yan.ma@intel.com>
Yan Ma committed
Aug 25, 2026 at 23:10 UTC
10ade939796611915ee481da076a897c0c54fcf3
1 file changed
+54
-44
docs/models/hardware_supported_models/xpu.md
+54
-44
@@ -10,53 +10,63 @@
10
11
### Text-only Language Models
12
13
-| Model | Architecture | BF16/FP16/Dynamic FP8 | Compressed_tensors FP8 | MXFP4 |
14
-| -------------------------------------------------- | ------------------------------------------------ | --------------------- | ---------------------- | ----- |
15
-| openai/gpt-oss-20b | GPTForCausalLM | | | ✅ |
16
-| openai/gpt-oss-120b | GPTForCausalLM | | | ✅ |
17
-| deepseek-ai/DeepSeek-R1-Distill-Llama-8B | LlamaForCausalLM | ✅ | | |
18
-| deepseek-ai/DeepSeek-R1-Distill-Qwen-14B | QwenForCausalLM | ✅ | | |
19
-| deepseek-ai/DeepSeek-R1-Distill-Qwen-32B | QwenForCausalLM | ✅ | | |
20
-| deepseek-ai/DeepSeek-R1-Distill-Llama-70B | LlamaForCausalLM | ✅ | | |
21
-| Qwen/Qwen2.5-72B-Instruct | Qwen2ForCausalLM | ✅ | | |
22
-| Qwen/Qwen3-14B | Qwen3ForCausalLM | ✅ | | |
23
-| Qwen/Qwen3-32B | Qwen3ForCausalLM | ✅ | | |
24
-| Qwen/Qwen3-30B-A3B | Qwen3ForCausalLM | ✅ | | |
25
-| Qwen/Qwen3-30B-A3B-GPTQ-Int4 | Qwen3ForCausalLM | ✅ | | |
26
-| Qwen/Qwen3-coder-30B-A3B-Instruct | Qwen3ForCausalLM | ✅ | | |
27
-| Qwen/QwQ-32B | QwenForCausalLM | ✅ | | |
28
-| deepseek-ai/DeepSeek-V2-Lite | DeepSeekForCausalLM | ✅ | | |
29
-| meta-llama/Llama-3.1-8B-Instruct | LlamaForCausalLM | ✅ | | |
30
-| THUDM/GLM-4-9B-chat | GLMForCausalLM | ✅ | | |
31
-| THUDM/CodeGeex4-All-9B | CodeGeexForCausalLM | ✅ | | |
32
-| chuhac/TeleChat2-35B | LlamaForCausalLM (TeleChat2 based on Llama arch) | ✅ | | |
33
-| 01-ai/Yi1.5-34B-Chat | YiForCausalLM | ✅ | | |
34
-| deepseek-ai/DeepSeek-Coder-33B-base | DeepSeekCoderForCausalLM | ✅ | | |
35
-| meta-llama/Llama-2-13b-chat-hf | LlamaForCausalLM | ✅ | | |
36
-| Qwen/Qwen1.5-14B-Chat | QwenForCausalLM | ✅ | | |
37
-| Qwen/Qwen1.5-32B-Chat | QwenForCausalLM | ✅ | | |
38
-| RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8-dynamic | LlamaForCausalLM | | ✅ | |
13
+Note: Online FP8 = [Online Quantization](https://docs.vllm.ai/en/latest/features/quantization/online/) to FP8.
14
+
15
+| Model | Architecture | Dtype |
16
+| ----------------------------------------------- | ------------------------------------------------ | --------------------- |
17
+| openai/gpt-oss-20b | GPTForCausalLM | MXFP4 |
18
+| openai/gpt-oss-120b | GPTForCausalLM | MXFP4 |
19
+| deepseek-ai/DeepSeek-R1-Distill-Llama-8B | LlamaForCausalLM | BF16/Online FP8 |
20
+| deepseek-ai/DeepSeek-R1-Distill-Qwen-14B | QwenForCausalLM | BF16/Online FP8 |
21
+| deepseek-ai/DeepSeek-R1-Distill-Qwen-32B | QwenForCausalLM | BF16/Online FP8 |
22
+| deepseek-ai/DeepSeek-R1-Distill-Llama-70B | LlamaForCausalLM | BF16/Online FP8 |
23
+| Qwen/Qwen2.5-72B-Instruct | Qwen2ForCausalLM | BF16/Online FP8 |
24
+| Qwen/Qwen3-14B | Qwen3ForCausalLM | BF16/Online FP8 |
25
+| Qwen/Qwen3-32B | Qwen3ForCausalLM | BF16/Online FP8 |
26
+| Qwen/Qwen3-30B-A3B | Qwen3ForCausalLM | BF16/Online FP8 |
27
+| Qwen/Qwen3-30B-A3B-FP8 | Qwen3ForCausalLM | FP8 |
28
+| Qwen/Qwen3-30B-A3B-GPTQ-Int4 | Qwen3ForCausalLM | Int4 |
29
+| Qwen/Qwen3-coder-30B-A3B-Instruct | Qwen3ForCausalLM | BF16/Online FP8 |
30
+| Qwen/Qwen3-Next-80B-A3B-Instruct | Qwen3NextForCausalLM | BF16/Online FP8 |
31
+| Qwen/Qwen3-Next-80B-A3B-Thinking | Qwen3NextForCausalLM | BF16/Online FP8 |
32
+| Qwen/QwQ-32B | QwenForCausalLM | BF16/Online FP8 |
33
+| deepseek-ai/DeepSeek-V2-Lite | DeepSeekForCausalLM | BF16/Online FP8 |
34
+| deepseek-ai/DeepSeek-V4-Flash | DeepseekV4ForCausalLM | BF16/Online FP8 |
35
+| meta-llama/Llama-3.1-8B-Instruct | LlamaForCausalLM | BF16/Online FP8 |
36
+| microsoft/Phi-3.5-mini-instruct | Phi3ForCausalLM | BF16/Online FP8 |
37
+| THUDM/GLM-4-9B-chat | GLMForCausalLM | BF16/Online FP8 |
38
+| THUDM/CodeGeex4-All-9B | CodeGeexForCausalLM | BF16/Online FP8 |
39
+| chuhac/TeleChat2-35B | LlamaForCausalLM (TeleChat2 based on Llama arch) | BF16/Online FP8 |
40
+| 01-ai/Yi1.5-34B-Chat | YiForCausalLM | BF16/Online FP8 |
41
+| deepseek-ai/DeepSeek-Coder-33B-base | DeepSeekCoderForCausalLM | BF16/Online FP8 |
42
+| meta-llama/Llama-2-13b-chat-hf | LlamaForCausalLM | FP16/Online FP8 |
43
+| Qwen/Qwen1.5-14B-Chat | QwenForCausalLM | BF16/Online FP8 |
44
+| Qwen/Qwen1.5-32B-Chat | QwenForCausalLM | BF16/Online FP8 |
45
+| RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8-dynamic | LlamaForCausalLM | FP8 |
46
47
### Multimodal Language Models
48
42
-| Model | Architecture | BF16 | Dynamic FP8 | MXFP4 |
43
-| ---------------------------- | -------------------------------- | ---- | ----------- | ----- |
44
-| OpenGVLab/InternVL3_5-8B | InternVLForConditionalGeneration | ✅ | ✅ | |
45
-| OpenGVLab/InternVL3_5-14B | InternVLForConditionalGeneration | ✅ | ✅ | |
46
-| OpenGVLab/InternVL3_5-38B | InternVLForConditionalGeneration | ✅ | ✅ | |
47
-| Qwen/Qwen2-VL-7B-Instruct | Qwen2VLForConditionalGeneration | ✅ | ✅ | |
48
-| Qwen/Qwen2.5-VL-72B-Instruct | Qwen2VLForConditionalGeneration | ✅ | ✅ | |
49
-| Qwen/Qwen2.5-VL-32B-Instruct | Qwen2VLForConditionalGeneration | ✅ | ✅ | |
50
-| THUDM/GLM-4v-9B | GLM4vForConditionalGeneration | ✅ | ✅ | |
51
-| openbmb/MiniCPM-V-4 | MiniCPMVForConditionalGeneration | ✅ | ✅ | |
49
+| Model | Architecture | Dtype |
50
+| ----------------------------- | ------------------------------------ | --------------- |
51
+| OpenGVLab/InternVL3_5-8B | InternVLForConditionalGeneration | BF16/Online FP8 |
52
+| OpenGVLab/InternVL3_5-14B | InternVLForConditionalGeneration | BF16/Online FP8 |
53
+| OpenGVLab/InternVL3_5-30B-A3B | InternVLForConditionalGeneration | BF16/Online FP8 |
54
+| OpenGVLab/InternVL3_5-38B | InternVLForConditionalGeneration | BF16/Online FP8 |
55
+| meta-models/Muse-Glimmer-30B | MuseGlimmerForConditionalGeneration | BF16/Online FP8 |
56
+| Qwen/Qwen2-VL-7B-Instruct | Qwen2VLForConditionalGeneration | BF16/Online FP8 |
57
+| Qwen/Qwen2.5-VL-72B-Instruct | Qwen2VLForConditionalGeneration | BF16/Online FP8 |
58
+| Qwen/Qwen2.5-VL-32B-Instruct | Qwen2VLForConditionalGeneration | BF16/Online FP8 |
59
+| Qwen/Qwen3-VL-32B-Instruct | Qwen3VLForConditionalGeneration | BF16/Online FP8 |
60
+| Qwen/Qwen3.5-35B-A3B | Qwen3_5MoeForConditionalGeneration | BF16/Online FP8 |
61
+| google/gemma-3-27b-it | Gemma3ForConditionalGeneration | BF16/Online FP8 |
62
+| google/gemma-4-31B-it | Gemma4ForConditionalGeneration | BF16/Online FP8 |
63
+| google/gemma-4-26B-A4B-it | Gemma4ForConditionalGeneration | BF16/Online FP8 |
64
+| THUDM/GLM-4v-9B | GLM4vForConditionalGeneration | BF16/Online FP8 |
65
+| openbmb/MiniCPM-V-4 | MiniCPMVForConditionalGeneration | BF16/Online FP8 |
66
67
### Embedding and Reranker Language Models
68
55
-| Model | Architecture | BF16 | Dynamic FP8 | MXFP4 |
56
-| ----------------------- | ------------------------------ | ---- | ----------- | ----- |
57
-| Qwen/Qwen3-Embedding-8B | Qwen3ForTextEmbedding | ✅ | ✅ | |
58
-| Qwen/Qwen3-Reranker-8B | Qwen3ForSequenceClassification | ✅ | ✅ | |
59
-
60
-✅ Runs and optimized.
61
-🟨 Runs and correct but not optimized to green yet.
62
-❌ Does not pass accuracy test or does not run.
69
+| Model | Architecture | Dtype |
70
+| ----------------------- | ------------------------------ | --------------- |
71
+| Qwen/Qwen3-Embedding-8B | Qwen3ForTextEmbedding | BF16/Online FP8 |
72
+| Qwen/Qwen3-Reranker-8B | Qwen3ForSequenceClassification | BF16/Online FP8 |