| 1 | # CPU - Intel® Xeon® |
| 2 | |
| 3 | !!! note "AMD Zen CPUs" |
| 4 | On AMD Zen 4 / Zen 5 CPUs, AMD Zen optimizations are auto-enabled when the [`zentorch`](https://github.com/amd/ZenDNN-pytorch-plugin) package is installed. All models supported by vLLM on CPU are supported on AMD Zen as well; model compatibility does not change. This page reflects the current CPU reference validation matrix on Intel systems. See [AMD Zen optimizations](../../getting_started/installation/cpu.md#amd-zen-optimizations) for details. |
| 5 | |
| 6 | ## Validated Hardware |
| 7 | |
| 8 | | Hardware | |
| 9 | | -------- | |
| 10 | | [Intel® Xeon® 6 Processors](https://www.intel.com/content/www/us/en/products/details/processors/xeon.html) | |
| 11 | | [Intel® Xeon® 5 Processors](https://www.intel.com/content/www/us/en/products/docs/processors/xeon/5th-gen-xeon-scalable-processors.html) | |
| 12 | |
| 13 | ## Deploy from a vLLM Recipe |
| 14 | |
| 15 | The **Recipe** column below links to published Xeon 6 configurations when |
| 16 | available. Open a recipe to review the latest deployment configuration for the |
| 17 | model and hardware. |
| 18 | |
| 19 | For one-step deployment with the pre-built CPU image, see |
| 20 | [Serve with vLLM Recipes](../../getting_started/installation/cpu.md#serve-with-vllm-recipes). |
| 21 | The image includes the Recipes tool, which can retrieve the latest published |
| 22 | recipe, apply Xeon hardware detection, and start `vllm serve`. |
| 23 | |
| 24 | For converter usage and interactive model/hardware discovery, see the |
| 25 | [Recipes tool documentation](../../../tools/recipes/README.md). |
| 26 | |
| 27 | ## Recommended Models |
| 28 | |
| 29 | ### Text-only Language Models |
| 30 | |
| 31 | | Model | Architecture | Supported | Recipe | |
| 32 | | ------------------------------------ | ---------------------------------------- | --------- | ------ | |
| 33 | | openai/gpt-oss-20b | GptOssForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/openai/gpt-oss-20b?hardware=xeon6) | |
| 34 | | meta-llama/Llama-3.1-8B | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.1-8B?hardware=xeon6) | |
| 35 | | meta-llama/Llama-3.1-8B-Instruct | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.1-8B-Instruct?hardware=xeon6) | |
| 36 | | meta-llama/Llama-3.2-1B | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.2-1B?hardware=xeon6) | |
| 37 | | meta-llama/Llama-3.2-1B-Instruct | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.2-1B-Instruct?hardware=xeon6) | |
| 38 | | meta-llama/Llama-3.2-3B-Instruct | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.2-3B-Instruct?hardware=xeon6) | |
| 39 | | meta-llama/Llama-3.3-70B-Instruct | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.3-70B-Instruct?hardware=xeon6) | |
| 40 | | RedHatAI/Meta-Llama-3.1-8B-quantized.w8a8 | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.1-8B?hardware=xeon6&variant=w8a8_int8) | |
| 41 | | RedHatAI/Meta-Llama-3.1-8B-Instruct-quantized.w8a8 | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.1-8B-Instruct?hardware=xeon6&variant=w8a8_int8) | |
| 42 | | RedHatAI/Llama-3.2-1B-Instruct-quantized.w8a8 | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.2-1B-Instruct?hardware=xeon6&variant=w8a8_int8) | |
| 43 | | RedHatAI/Llama-3.2-3B-Instruct-quantized.w8a8 | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.2-3B-Instruct?hardware=xeon6&variant=w8a8_int8) | |
| 44 | | RedHatAI/DeepSeek-R1-Distill-Llama-70B-quantized.w8a8 | LlamaForCausalLM | ✅ | — | |
| 45 | | hugging-quants/Meta-Llama-3.1-8B-Instruct-AWQ-INT4 | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.1-8B-Instruct?hardware=xeon6&variant=awq_int4) | |
| 46 | | AMead10/Llama-3.2-1B-Instruct-AWQ | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.2-1B-Instruct?hardware=xeon6&variant=awq_int4) | |
| 47 | | AMead10/Llama-3.2-3B-Instruct-AWQ | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.2-3B-Instruct?hardware=xeon6&variant=awq_int4) | |
| 48 | | TheBloke/TinyLlama-1.1B-Chat-v1.0-AWQ | LlamaForCausalLM | ✅ | — | |
| 49 | | TheBloke/TinyLlama-1.1B-Chat-v1.0-GPTQ | LlamaForCausalLM | ✅ | — | |
| 50 | | ibm-granite/granite-3.2-2b-instruct | GraniteForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/ibm-granite/granite-3.2-2b-instruct?hardware=xeon6) | |
| 51 | | Qwen/Qwen3-1.7B | Qwen3ForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/Qwen/Qwen3-1.7B?hardware=xeon6) | |
| 52 | | Qwen/Qwen3-4B | Qwen3ForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/Qwen/Qwen3-4B?hardware=xeon6) | |
| 53 | | Qwen/Qwen3-8B | Qwen3ForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/Qwen/Qwen3-8B?hardware=xeon6) | |
| 54 | | Qwen/Qwen3-14B | Qwen3ForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/Qwen/Qwen3-14B?hardware=xeon6) | |
| 55 | | Qwen/Qwen3-14B-FP8 | Qwen3ForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/Qwen/Qwen3-14B?hardware=xeon6&variant=fp8) | |
| 56 | | Qwen/Qwen3-14B-AWQ | Qwen3ForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/Qwen/Qwen3-14B?hardware=xeon6&variant=awq) | |
| 57 | | Qwen/Qwen3-30B-A3B | Qwen3MoeForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/Qwen/Qwen3-30B-A3B?hardware=xeon6) | |
| 58 | | Qwen/Qwen3-30B-A3B-Instruct-2507-FP8 | Qwen3MoeForCausalLM | ✅ | — | |
| 59 | | Qwen/QwQ-32B | Qwen2ForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/Qwen/QwQ-32B?hardware=xeon6) | |
| 60 | | Qwen/QwQ-32B-AWQ | Qwen2ForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/Qwen/QwQ-32B?hardware=xeon6&variant=awq) | |
| 61 | | Qwen/Qwen1.5-0.5B-Chat-GPTQ-Int4 | Qwen2ForCausalLM | ✅ | — | |
| 62 | | RedHatAI/QwQ-32B-quantized.w8a8 | Qwen2ForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/Qwen/QwQ-32B?hardware=xeon6&variant=w8a8_int8) | |
| 63 | | zai-org/glm-4-9b-hf | GLMForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/zai-org/glm-4-9b-hf?hardware=xeon6) | |
| 64 | | google/gemma-7b | GemmaForCausalLM | ✅ | — | |
| 65 | | microsoft/Phi-4-reasoning | Phi3ForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/microsoft/Phi-4-reasoning?hardware=xeon6) | |
| 66 | | mistralai/Mistral-7B-Instruct-v0.2 | MistralForCausalLM | ✅ | — | |
| 67 | | TheBloke/Mistral-7B-Instruct-v0.2-AWQ | MistralForCausalLM | ✅ | — | |
| 68 | |
| 69 | ### Multimodal Language Models |
| 70 | |
| 71 | | Model | Architecture | Supported | Recipe | |
| 72 | | ------------------------------------ | ---------------------------------------- | --------- | ------ | |
| 73 | | meta-llama/Llama-4-Scout-17B-16E-Instruct | Llama4ForConditionalGeneration | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-4-Scout-17B-16E-Instruct?hardware=xeon6) | |
| 74 | | google/gemma-3-4b-it | Gemma3ForConditionalGeneration | ✅ | — | |
| 75 | | google/gemma-3-12b-it | Gemma3ForConditionalGeneration | ✅ | — | |
| 76 | | google/gemma-4-E4B-it | Gemma4ForConditionalGeneration | ✅ | [Xeon 6](https://recipes.vllm.ai/Google/gemma-4-E4B-it?hardware=xeon6) | |
| 77 | | google/gemma-4-E2B-it | Gemma4ForConditionalGeneration | ✅ | [Xeon 6](https://recipes.vllm.ai/Google/gemma-4-E2B-it?hardware=xeon6) | |
| 78 | | google/gemma-4-26B-A4B-it | Gemma4ForConditionalGeneration | ✅ | [Xeon 6](https://recipes.vllm.ai/Google/gemma-4-26B-A4B-it?hardware=xeon6) | |
| 79 | | microsoft/Phi-4-multimodal-instruct | Phi4MMForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/microsoft/Phi-4-multimodal-instruct?hardware=xeon6) | |
| 80 | | Qwen/Qwen2.5-VL-7B-Instruct | Qwen2VLForConditionalGeneration | ✅ | [Xeon 6](https://recipes.vllm.ai/Qwen/Qwen2.5-VL-7B-Instruct?hardware=xeon6) | |
| 81 | | Qwen/Qwen3-VL-30B-A3B-Instruct | Qwen3VLMoeForConditionalGeneration | ✅ | [Xeon 6](https://recipes.vllm.ai/Qwen/Qwen3-VL-30B-A3B-Instruct?hardware=xeon6) | |
| 82 | | openai/whisper-large-v3 | WhisperForConditionalGeneration | ✅ | [Xeon 6](https://recipes.vllm.ai/openai/whisper-large-v3?hardware=xeon6) | |
| 83 | |
| 84 | ✅ Runs and optimized. |