A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 12, 12:15 UTC
add xpu supported model and model id for cpu (#29380)
Signed-off-by: Tsai, Louie <louie.tsai@intel.com>
Louie Tsai committed
Nov 26, 2025 at 19:30 UTC
9bb33c8919024a50e48687c08d706df3fa3302ed
2 files changed
+82
-9
docs/models/hardware_supported_models/cpu.md
+17
-9
@@ -1,25 +1,33 @@
1
# CPU - Intel® Xeon®
2
3
+## Validated Hardware
4
+
5
+| Hardware |
6
+| ----------------------------------------- |
7
+| [Intel® Xeon® 6 Processors](https://www.intel.com/content/www/us/en/products/details/processors/xeon.html) |
8
+| [Intel® Xeon® 5 Processors](https://www.intel.com/content/www/us/en/products/docs/processors/xeon/5th-gen-xeon-scalable-processors.html) |
9
+
10
## Supported Models
11
12
### Text-only Language Models
13
14
| Model | Architecture | Supported |
15
|--------------------------------------|-------------------------------------------|-----------|
9
-| meta-llama/Llama-3.1 / 3.3 | LlamaForCausalLM | ✅ |
10
-| meta-llama/Llama-4-Scout | Llama4ForConditionalGeneration | ✅ |
11
-| meta-llama/Llama-4-Maverick | Llama4ForConditionalGeneration | ✅ |
12
-| ibm-granite/granite (Granite-MOE) | GraniteMoeForCausalLM | ✅ |
13
-| Qwen/Qwen3 | Qwen3ForCausalLM | ✅ |
14
-| zai-org/GLM-4.5 | GLMForCausalLM | ✅ |
15
-| google/gemma | GemmaForCausalLM | ✅ |
16
+| meta-llama/Llama-3.1-8B-Instruct | LlamaForCausalLM | ✅ |
17
+| meta-llama/Llama-3.2-3B-Instruct | LlamaForCausalLM | ✅ |
18
+| ibm-granite/granite-3.2-2b-instruct | GraniteForCausalLM | ✅ |
19
+| Qwen/Qwen3-1.7B | Qwen3ForCausalLM | ✅ |
20
+| Qwen/Qwen3-4B | Qwen3ForCausalLM | ✅ |
21
+| Qwen/Qwen3-8B | Qwen3ForCausalLM | ✅ |
22
+| zai-org/glm-4-9b-hf | GLMForCausalLM | ✅ |
23
+| google/gemma-7b | GemmaForCausalLM | ✅ |
24
25
### Multimodal Language Models
26
27
| Model | Architecture | Supported |
28
|--------------------------------------|-------------------------------------------|-----------|
21
-| Qwen/Qwen2.5-VL | Qwen2VLForConditionalGeneration | ✅ |
22
-| openai/whisper | WhisperForConditionalGeneration | ✅ |
29
+| Qwen/Qwen2.5-VL-7B-Instruct | Qwen2VLForConditionalGeneration | ✅ |
30
+| openai/whisper-large-v3 | WhisperForConditionalGeneration | ✅ |
31
32
✅ Runs and optimized.
33
🟨 Runs and correct but not optimized to green yet.
docs/models/hardware_supported_models/xpu.md
new
+65
@@ -0,0 +1,65 @@
1
+# XPU - Intel® GPUs
2
+
3
+## Validated Hardware
4
+
5
+| Hardware |
6
+| ----------------------------------------- |
7
+| [Intel® Arc™ Pro B-Series Graphics](https://www.intel.com/content/www/us/en/products/docs/discrete-gpus/arc/workstations/b-series/overview.html) |
8
+
9
+## Supported Models
10
+
11
+### Text-only Language Models
12
+
13
+| Model | Architecture | FP16 | Dynamic FP8 | MXFP4 |
14
+| ----------------------------------------- | ---------------------------------------------------- | ---- | ----------- | ----- |
15
+| openai/gpt-oss-20b | GPTForCausalLM | | | ✅ |
16
+| openai/gpt-oss-120b | GPTForCausalLM | | | ✅ |
17
+| deepseek-ai/DeepSeek-R1-Distill-Llama-8B | LlamaForCausalLM | ✅ | ✅ | |
18
+| deepseek-ai/DeepSeek-R1-Distill-Qwen-14B | QwenForCausalLM | ✅ | ✅ | |
19
+| deepseek-ai/DeepSeek-R1-Distill-Qwen-32B | QwenForCausalLM | ✅ | ✅ | |
20
+| deepseek-ai/DeepSeek-R1-Distill-Llama-70B | LlamaForCausalLM | ✅ | ✅ | |
21
+| Qwen/Qwen2.5-72B-Instruct | Qwen2ForCausalLM | ✅ | ✅ | |
22
+| Qwen/Qwen3-14B | Qwen3ForCausalLM | ✅ | ✅ | |
23
+| Qwen/Qwen3-32B | Qwen3ForCausalLM | ✅ | ✅ | |
24
+| Qwen/Qwen3-30B-A3B | Qwen3ForCausalLM | ✅ | ✅ | |
25
+| Qwen/Qwen3-30B-A3B-GPTQ-Int4 | Qwen3ForCausalLM | ✅ | ✅ | |
26
+| Qwen/Qwen3-coder-30B-A3B-Instruct | Qwen3ForCausalLM | ✅ | ✅ | |
27
+| Qwen/QwQ-32B | QwenForCausalLM | ✅ | ✅ | |
28
+| deepseek-ai/DeepSeek-V2-Lite | DeepSeekForCausalLM | ✅ | ✅ | |
29
+| meta-llama/Llama-3.1-8B-Instruct | LlamaForCausalLM | ✅ | ✅ | |
30
+| baichuan-inc/Baichuan2-13B-Chat | BaichuanForCausalLM | ✅ | ✅ | |
31
+| THUDM/GLM-4-9B-chat | GLMForCausalLM | ✅ | ✅ | |
32
+| THUDM/CodeGeex4-All-9B | CodeGeexForCausalLM | ✅ | ✅ | |
33
+| chuhac/TeleChat2-35B | LlamaForCausalLM (TeleChat2 based on Llama arch) | ✅ | ✅ | |
34
+| 01-ai/Yi1.5-34B-Chat | YiForCausalLM | ✅ | ✅ | |
35
+| THUDM/CodeGeex4-All-9B | CodeGeexForCausalLM | ✅ | ✅ | |
36
+| deepseek-ai/DeepSeek-Coder-33B-base | DeepSeekCoderForCausalLM | ✅ | ✅ | |
37
+| baichuan-inc/Baichuan2-13B-Chat | BaichuanForCausalLM | ✅ | ✅ | |
38
+| meta-llama/Llama-2-13b-chat-hf | LlamaForCausalLM | ✅ | ✅ | |
39
+| THUDM/CodeGeex4-All-9B | CodeGeexForCausalLM | ✅ | ✅ | |
40
+| Qwen/Qwen1.5-14B-Chat | QwenForCausalLM | ✅ | ✅ | |
41
+| Qwen/Qwen1.5-32B-Chat | QwenForCausalLM | ✅ | ✅ | |
42
+
43
+### Multimodal Language Models
44
+
45
+| Model | Architecture | FP16 | Dynamic FP8 | MXFP4 |
46
+| ---------------------------- | -------------------------------- | ---- | ----------- | ----- |
47
+| OpenGVLab/InternVL3_5-8B | InternVLForConditionalGeneration | ✅ | ✅ | |
48
+| OpenGVLab/InternVL3_5-14B | InternVLForConditionalGeneration | ✅ | ✅ | |
49
+| OpenGVLab/InternVL3_5-38B | InternVLForConditionalGeneration | ✅ | ✅ | |
50
+| Qwen/Qwen2-VL-7B-Instruct | Qwen2VLForConditionalGeneration | ✅ | ✅ | |
51
+| Qwen/Qwen2.5-VL-72B-Instruct | Qwen2VLForConditionalGeneration | ✅ | ✅ | |
52
+| Qwen/Qwen2.5-VL-32B-Instruct | Qwen2VLForConditionalGeneration | ✅ | ✅ | |
53
+| THUDM/GLM-4v-9B | GLM4vForConditionalGeneration | ✅ | ✅ | |
54
+| openbmb/MiniCPM-V-4 | MiniCPMVForConditionalGeneration | ✅ | ✅ | |
55
+
56
+### Embedding and Reranker Language Models
57
+
58
+| Model | Architecture | FP16 | Dynamic FP8 | MXFP4 |
59
+| ----------------------- | ------------------------------ | ---- | ----------- | ----- |
60
+| Qwen/Qwen3-Embedding-8B | Qwen3ForTextEmbedding | ✅ | ✅ | |
61
+| Qwen/Qwen3-Reranker-8B | Qwen3ForSequenceClassification | ✅ | ✅ | |
62
+
63
+✅ Runs and optimized.
64
+🟨 Runs and correct but not optimized to green yet.
65
+❌ Does not pass accuracy test or does not run.