@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 4f6fa6341d
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 4, 01:15 UTC

[XPU] update supported models on XPU (#41911)

Signed-off-by: Yan Ma <yan.ma@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>

Yan Ma committed May 9, 2026 at 10:44 UTC 4f6fa6341da0789714b240c7d6baf53a5106f5e2
1 file changed +32 -31
docs/models/hardware_supported_models/xpu.md
+32 -31
@@ -10,39 +10,40 @@
10
11 ### Text-only Language Models
12
13 -| Model | Architecture | FP16 | Dynamic FP8 | MXFP4 |
14 -| ----------------------------------------- | ---------------------------------------------------- | ---- | ----------- | ----- |
15 -| openai/gpt-oss-20b | GPTForCausalLM | | | ✅ |
16 -| openai/gpt-oss-120b | GPTForCausalLM | | | ✅ |
17 -| deepseek-ai/DeepSeek-R1-Distill-Llama-8B | LlamaForCausalLM | ✅ | ✅ | |
18 -| deepseek-ai/DeepSeek-R1-Distill-Qwen-14B | QwenForCausalLM | ✅ | ✅ | |
19 -| deepseek-ai/DeepSeek-R1-Distill-Qwen-32B | QwenForCausalLM | ✅ | ✅ | |
20 -| deepseek-ai/DeepSeek-R1-Distill-Llama-70B | LlamaForCausalLM | ✅ | ✅ | |
21 -| Qwen/Qwen2.5-72B-Instruct | Qwen2ForCausalLM | ✅ | ✅ | |
22 -| Qwen/Qwen3-14B | Qwen3ForCausalLM | ✅ | ✅ | |
23 -| Qwen/Qwen3-32B | Qwen3ForCausalLM | ✅ | ✅ | |
24 -| Qwen/Qwen3-30B-A3B | Qwen3ForCausalLM | ✅ | ✅ | |
25 -| Qwen/Qwen3-30B-A3B-GPTQ-Int4 | Qwen3ForCausalLM | ✅ | ✅ | |
26 -| Qwen/Qwen3-coder-30B-A3B-Instruct | Qwen3ForCausalLM | ✅ | ✅ | |
27 -| Qwen/QwQ-32B | QwenForCausalLM | ✅ | ✅ | |
28 -| deepseek-ai/DeepSeek-V2-Lite | DeepSeekForCausalLM | ✅ | ✅ | |
29 -| meta-llama/Llama-3.1-8B-Instruct | LlamaForCausalLM | ✅ | ✅ | |
30 -| baichuan-inc/Baichuan2-13B-Chat | BaichuanForCausalLM | ✅ | ✅ | |
31 -| THUDM/GLM-4-9B-chat | GLMForCausalLM | ✅ | ✅ | |
32 -| THUDM/CodeGeex4-All-9B | CodeGeexForCausalLM | ✅ | ✅ | |
33 -| chuhac/TeleChat2-35B | LlamaForCausalLM (TeleChat2 based on Llama arch) | ✅ | ✅ | |
34 -| 01-ai/Yi1.5-34B-Chat | YiForCausalLM | ✅ | ✅ | |
35 -| THUDM/CodeGeex4-All-9B | CodeGeexForCausalLM | ✅ | ✅ | |
36 -| deepseek-ai/DeepSeek-Coder-33B-base | DeepSeekCoderForCausalLM | ✅ | ✅ | |
37 -| baichuan-inc/Baichuan2-13B-Chat | BaichuanForCausalLM | ✅ | ✅ | |
38 -| meta-llama/Llama-2-13b-chat-hf | LlamaForCausalLM | ✅ | ✅ | |
39 -| THUDM/CodeGeex4-All-9B | CodeGeexForCausalLM | ✅ | ✅ | |
40 -| Qwen/Qwen1.5-14B-Chat | QwenForCausalLM | ✅ | ✅ | |
41 -| Qwen/Qwen1.5-32B-Chat | QwenForCausalLM | ✅ | ✅ | |
13 +| Model | Architecture | BF16/FP16/Dynamic FP8 | Compressed_tensors FP8 | MXFP4 |
14 +| -------------------------------------------------- | ------------------------------------------------ | --------------------- | ---------------------- | ----- |
15 +| openai/gpt-oss-20b | GPTForCausalLM | | | ✅ |
16 +| openai/gpt-oss-120b | GPTForCausalLM | | | ✅ |
17 +| deepseek-ai/DeepSeek-R1-Distill-Llama-8B | LlamaForCausalLM | ✅ | | |
18 +| deepseek-ai/DeepSeek-R1-Distill-Qwen-14B | QwenForCausalLM | ✅ | | |
19 +| deepseek-ai/DeepSeek-R1-Distill-Qwen-32B | QwenForCausalLM | ✅ | | |
20 +| deepseek-ai/DeepSeek-R1-Distill-Llama-70B | LlamaForCausalLM | ✅ | | |
21 +| Qwen/Qwen2.5-72B-Instruct | Qwen2ForCausalLM | ✅ | | |
22 +| Qwen/Qwen3-14B | Qwen3ForCausalLM | ✅ | | |
23 +| Qwen/Qwen3-32B | Qwen3ForCausalLM | ✅ | | |
24 +| Qwen/Qwen3-30B-A3B | Qwen3ForCausalLM | ✅ | | |
25 +| Qwen/Qwen3-30B-A3B-GPTQ-Int4 | Qwen3ForCausalLM | ✅ | | |
26 +| Qwen/Qwen3-coder-30B-A3B-Instruct | Qwen3ForCausalLM | ✅ | | |
27 +| Qwen/QwQ-32B | QwenForCausalLM | ✅ | | |
28 +| deepseek-ai/DeepSeek-V2-Lite | DeepSeekForCausalLM | ✅ | | |
29 +| meta-llama/Llama-3.1-8B-Instruct | LlamaForCausalLM | ✅ | | |
30 +| baichuan-inc/Baichuan2-13B-Chat | BaichuanForCausalLM | ✅ | | |
31 +| THUDM/GLM-4-9B-chat | GLMForCausalLM | ✅ | | |
32 +| THUDM/CodeGeex4-All-9B | CodeGeexForCausalLM | ✅ | | |
33 +| chuhac/TeleChat2-35B | LlamaForCausalLM (TeleChat2 based on Llama arch) | ✅ | | |
34 +| 01-ai/Yi1.5-34B-Chat | YiForCausalLM | ✅ | | |
35 +| THUDM/CodeGeex4-All-9B | CodeGeexForCausalLM | ✅ | | |
36 +| deepseek-ai/DeepSeek-Coder-33B-base | DeepSeekCoderForCausalLM | ✅ | | |
37 +| baichuan-inc/Baichuan2-13B-Chat | BaichuanForCausalLM | ✅ | | |
38 +| meta-llama/Llama-2-13b-chat-hf | LlamaForCausalLM | ✅ | | |
39 +| THUDM/CodeGeex4-All-9B | CodeGeexForCausalLM | ✅ | | |
40 +| Qwen/Qwen1.5-14B-Chat | QwenForCausalLM | ✅ | | |
41 +| Qwen/Qwen1.5-32B-Chat | QwenForCausalLM | ✅ | | |
42 +| RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8-dynamic | LlamaForCausalLM | | ✅ | |
43
44 ### Multimodal Language Models
45
45 -| Model | Architecture | FP16 | Dynamic FP8 | MXFP4 |
46 +| Model | Architecture | BF16 | Dynamic FP8 | MXFP4 |
47 | ---------------------------- | -------------------------------- | ---- | ----------- | ----- |
48 | OpenGVLab/InternVL3_5-8B | InternVLForConditionalGeneration | ✅ | ✅ | |
49 | OpenGVLab/InternVL3_5-14B | InternVLForConditionalGeneration | ✅ | ✅ | |
@@ -55,7 +56,7 @@
56
57 ### Embedding and Reranker Language Models
58
58 -| Model | Architecture | FP16 | Dynamic FP8 | MXFP4 |
59 +| Model | Architecture | BF16 | Dynamic FP8 | MXFP4 |
60 | ----------------------- | ------------------------------ | ---- | ----------- | ----- |
61 | Qwen/Qwen3-Embedding-8B | Qwen3ForTextEmbedding | ✅ | ✅ | |
62 | Qwen/Qwen3-Reranker-8B | Qwen3ForSequenceClassification | ✅ | ✅ | |