xpu.md · tduncan/Nvidia-Optimization-Stack-VLLM

@tduncan / Nvidia-Optimization-Stack-VLLM

Code Commits Issues Pulls Cloud Agent CI/CD

main

md 72 lines 6.12 KB

Rendered Raw

1	# XPU - Intel® GPUs
2
3	## Validated Hardware
4
5	\| Hardware \|
6	\| -------- \|
7	\| [Intel® Arc™ Pro B-Series Graphics](https://www.intel.com/content/www/us/en/products/docs/discrete-gpus/arc/workstations/b-series/overview.html) \|
8
9	## Recommended Models
10
11	### Text-only Language Models
12
13	Note: Online FP8 = [Online Quantization](https://docs.vllm.ai/en/latest/features/quantization/online/) to FP8.
14
15	\| Model \| Architecture \| Dtype \|
16	\| ----------------------------------------------- \| ------------------------------------------------ \| --------------------- \|
17	\| openai/gpt-oss-20b \| GPTForCausalLM \| MXFP4 \|
18	\| openai/gpt-oss-120b \| GPTForCausalLM \| MXFP4 \|
19	\| deepseek-ai/DeepSeek-R1-Distill-Llama-8B \| LlamaForCausalLM \| BF16/Online FP8 \|
20	\| deepseek-ai/DeepSeek-R1-Distill-Qwen-14B \| QwenForCausalLM \| BF16/Online FP8 \|
21	\| deepseek-ai/DeepSeek-R1-Distill-Qwen-32B \| QwenForCausalLM \| BF16/Online FP8 \|
22	\| deepseek-ai/DeepSeek-R1-Distill-Llama-70B \| LlamaForCausalLM \| BF16/Online FP8 \|
23	\| Qwen/Qwen2.5-72B-Instruct \| Qwen2ForCausalLM \| BF16/Online FP8 \|
24	\| Qwen/Qwen3-14B \| Qwen3ForCausalLM \| BF16/Online FP8 \|
25	\| Qwen/Qwen3-32B \| Qwen3ForCausalLM \| BF16/Online FP8 \|
26	\| Qwen/Qwen3-30B-A3B \| Qwen3ForCausalLM \| BF16/Online FP8 \|
27	\| Qwen/Qwen3-30B-A3B-FP8 \| Qwen3ForCausalLM \| FP8 \|
28	\| Qwen/Qwen3-30B-A3B-GPTQ-Int4 \| Qwen3ForCausalLM \| Int4 \|
29	\| Qwen/Qwen3-coder-30B-A3B-Instruct \| Qwen3ForCausalLM \| BF16/Online FP8 \|
30	\| Qwen/Qwen3-Next-80B-A3B-Instruct \| Qwen3NextForCausalLM \| BF16/Online FP8 \|
31	\| Qwen/Qwen3-Next-80B-A3B-Thinking \| Qwen3NextForCausalLM \| BF16/Online FP8 \|
32	\| Qwen/QwQ-32B \| QwenForCausalLM \| BF16/Online FP8 \|
33	\| deepseek-ai/DeepSeek-V2-Lite \| DeepSeekForCausalLM \| BF16/Online FP8 \|
34	\| deepseek-ai/DeepSeek-V4-Flash \| DeepseekV4ForCausalLM \| BF16/Online FP8 \|
35	\| meta-llama/Llama-3.1-8B-Instruct \| LlamaForCausalLM \| BF16/Online FP8 \|
36	\| microsoft/Phi-3.5-mini-instruct \| Phi3ForCausalLM \| BF16/Online FP8 \|
37	\| THUDM/GLM-4-9B-chat \| GLMForCausalLM \| BF16/Online FP8 \|
38	\| THUDM/CodeGeex4-All-9B \| CodeGeexForCausalLM \| BF16/Online FP8 \|
39	\| chuhac/TeleChat2-35B \| LlamaForCausalLM (TeleChat2 based on Llama arch) \| BF16/Online FP8 \|
40	\| 01-ai/Yi1.5-34B-Chat \| YiForCausalLM \| BF16/Online FP8 \|
41	\| deepseek-ai/DeepSeek-Coder-33B-base \| DeepSeekCoderForCausalLM \| BF16/Online FP8 \|
42	\| meta-llama/Llama-2-13b-chat-hf \| LlamaForCausalLM \| FP16/Online FP8 \|
43	\| Qwen/Qwen1.5-14B-Chat \| QwenForCausalLM \| BF16/Online FP8 \|
44	\| Qwen/Qwen1.5-32B-Chat \| QwenForCausalLM \| BF16/Online FP8 \|
45	\| RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8-dynamic \| LlamaForCausalLM \| FP8 \|
46
47	### Multimodal Language Models
48
49	\| Model \| Architecture \| Dtype \|
50	\| ----------------------------- \| ------------------------------------ \| --------------- \|
51	\| OpenGVLab/InternVL3_5-8B \| InternVLForConditionalGeneration \| BF16/Online FP8 \|
52	\| OpenGVLab/InternVL3_5-14B \| InternVLForConditionalGeneration \| BF16/Online FP8 \|
53	\| OpenGVLab/InternVL3_5-30B-A3B \| InternVLForConditionalGeneration \| BF16/Online FP8 \|
54	\| OpenGVLab/InternVL3_5-38B \| InternVLForConditionalGeneration \| BF16/Online FP8 \|
55	\| meta-models/Muse-Glimmer-30B \| MuseGlimmerForConditionalGeneration \| BF16/Online FP8 \|
56	\| Qwen/Qwen2-VL-7B-Instruct \| Qwen2VLForConditionalGeneration \| BF16/Online FP8 \|
57	\| Qwen/Qwen2.5-VL-72B-Instruct \| Qwen2VLForConditionalGeneration \| BF16/Online FP8 \|
58	\| Qwen/Qwen2.5-VL-32B-Instruct \| Qwen2VLForConditionalGeneration \| BF16/Online FP8 \|
59	\| Qwen/Qwen3-VL-32B-Instruct \| Qwen3VLForConditionalGeneration \| BF16/Online FP8 \|
60	\| Qwen/Qwen3.5-35B-A3B \| Qwen3_5MoeForConditionalGeneration \| BF16/Online FP8 \|
61	\| google/gemma-3-27b-it \| Gemma3ForConditionalGeneration \| BF16/Online FP8 \|
62	\| google/gemma-4-31B-it \| Gemma4ForConditionalGeneration \| BF16/Online FP8 \|
63	\| google/gemma-4-26B-A4B-it \| Gemma4ForConditionalGeneration \| BF16/Online FP8 \|
64	\| THUDM/GLM-4v-9B \| GLM4vForConditionalGeneration \| BF16/Online FP8 \|
65	\| openbmb/MiniCPM-V-4 \| MiniCPMVForConditionalGeneration \| BF16/Online FP8 \|
66
67	### Embedding and Reranker Language Models
68
69	\| Model \| Architecture \| Dtype \|
70	\| ----------------------- \| ------------------------------ \| --------------- \|
71	\| Qwen/Qwen3-Embedding-8B \| Qwen3ForTextEmbedding \| BF16/Online FP8 \|
72	\| Qwen/Qwen3-Reranker-8B \| Qwen3ForSequenceClassification \| BF16/Online FP8 \|