A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 9, 16:00 UTC
Update CPU doc according to feedback (#32963)
Signed-off-by: Tsai, Louie <louie.tsai@intel.com> Signed-off-by: Louie Tsai <louie.tsai@intel.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Louie Tsai committed
Jan 24, 2026 at 08:02 UTC
719ac592edffd5748ca43ebf66d9311b0e132b24
3 files changed
+4
-4
docs/benchmarking/dashboard.md
+2
-2
@@ -13,14 +13,14 @@ For x86 CPU environment, please use the image with "-cpu" postfix. For AArch64 C
13
Here is an example for docker run command for CPU. For GPUs skip setting the `ON_CPU` env var.
14
15
```bash
16
-export VLLM_COMMIT=1da94e673c257373280026f75ceb4effac80e892 # use full commit hash from the main branch
16
+export VLLM_COMMIT=7f42dc20bb2800d09faa72b26f25d54e26f1b694 # use full commit hash from the main branch
17
export HF_TOKEN=<valid Hugging Face token>
18
if [[ "$(uname -m)" == aarch64 || "$(uname -m)" == arm64 ]]; then
19
IMG_SUFFIX="arm64-cpu"
20
else
21
IMG_SUFFIX="cpu"
22
fi
23
-docker run -it --entrypoint /bin/bash -v /data/huggingface:/root/.cache/huggingface -e HF_TOKEN=$HF_TOKEN -e ON_ARM64_CPU=1 --shm-size=16g --name vllm-cpu-ci public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:${VLLM_COMMIT}-${IMG_SUFFIX}
23
+docker run -it --entrypoint /bin/bash -v /data/huggingface:/root/.cache/huggingface -e HF_TOKEN=$HF_TOKEN -e ON_CPU=1 --shm-size=16g --name vllm-cpu-ci public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:${VLLM_COMMIT}-${IMG_SUFFIX}
24
```
25
26
Then, run below command inside the docker instance.
docs/models/hardware_supported_models/cpu.md
+1
-1
@@ -7,7 +7,7 @@
7
| [Intel® Xeon® 6 Processors](https://www.intel.com/content/www/us/en/products/details/processors/xeon.html) |
8
| [Intel® Xeon® 5 Processors](https://www.intel.com/content/www/us/en/products/docs/processors/xeon/5th-gen-xeon-scalable-processors.html) |
9
10
-## Supported Models
10
+## Recommended Models
11
12
### Text-only Language Models
13
docs/models/hardware_supported_models/xpu.md
+1
-1
@@ -6,7 +6,7 @@
6
| ----------------------------------------- |
7
| [Intel® Arc™ Pro B-Series Graphics](https://www.intel.com/content/www/us/en/products/docs/discrete-gpus/arc/workstations/b-series/overview.html) |
8
9
-## Supported Models
9
+## Recommended Models
10
11
### Text-only Language Models
12