A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[xpu] upgrade ipex/python3.12 for xpu (#23830)
Signed-off-by: Yan Ma <yan.ma@intel.com>
Yan Ma committed
Sep 8, 2025 at 10:07 UTC
67841317d11653febdb321d7748f3d7e0c242d64
3 files changed
+34
-25
docker/Dockerfile.xpu
+22
-11
@@ -1,12 +1,10 @@
1
FROM intel/deep-learning-essentials:2025.1.3-0-devel-ubuntu24.04 AS vllm-base
2
3
-RUN rm /etc/apt/sources.list.d/intel-graphics.list
3
+RUN wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB | gpg --dearmor | tee /usr/share/keyrings/oneapi-archive-keyring.gpg > /dev/null && \
4
+ echo "deb [signed-by=/usr/share/keyrings/oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main" | tee /etc/apt/sources.list.d/oneAPI.list && \
5
+ add-apt-repository -y ppa:kobuk-team/intel-graphics
6
7
RUN apt clean && apt-get update -y && \
6
- apt-get install -y software-properties-common && \
7
- add-apt-repository ppa:deadsnakes/ppa && \
8
- apt-get install -y python3.10 python3.10-distutils && \
9
- curl -sS https://bootstrap.pypa.io/get-pip.py | python3.10 && \
8
apt-get install -y --no-install-recommends --fix-missing \
9
curl \
10
ffmpeg \
@@ -17,17 +15,29 @@ RUN apt clean && apt-get update -y && \
15
libgl1 \
16
lsb-release \
17
numactl \
20
- python3.10-dev \
21
- wget
18
+ wget \
19
+ vim \
20
+ python3.12 \
21
+ python3.12-dev \
22
+ python3-pip
23
24
+RUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.12 1
25
+RUN update-alternatives --install /usr/bin/python python /usr/bin/python3.12 1
26
24
-RUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.10 1
25
-RUN update-alternatives --install /usr/bin/python python /usr/bin/python3.10 1
27
+RUN apt install -y libze1 libze-dev libze-intel-gpu1 intel-opencl-icd libze-intel-gpu-raytracing
28
+
29
+RUN wget https://github.com/uxlfoundation/oneCCL/releases/download/2021.15.4/intel-oneccl-2021.15.4.11_offline.sh
30
+RUN bash intel-oneccl-2021.15.4.11_offline.sh -a --silent --eula accept && echo "source /opt/intel/oneapi/setvars.sh --force" >> /root/.bashrc
31
+SHELL ["bash", "-c"]
32
+CMD ["bash", "-c", "source /root/.bashrc && exec bash"]
33
34
WORKDIR /workspace/vllm
35
COPY requirements/xpu.txt /workspace/vllm/requirements/xpu.txt
36
COPY requirements/common.txt /workspace/vllm/requirements/common.txt
37
38
+# suppress the python externally managed environment error
39
+RUN python3 -m pip config set global.break-system-packages true
40
+
41
RUN --mount=type=cache,target=/root/.cache/pip \
42
pip install --no-cache-dir \
43
-r requirements/xpu.txt
@@ -54,8 +64,9 @@ FROM vllm-base AS vllm-openai
64
RUN --mount=type=cache,target=/root/.cache/pip \
65
pip install accelerate hf_transfer pytest pytest_asyncio lm_eval[api] modelscope
66
57
-ENV VLLM_USAGE_SOURCE production-docker-image \
58
- TRITON_XPU_PROFILE 1
67
+RUN --mount=type=cache,target=/root/.cache/pip \
68
+ pip uninstall oneccl oneccl-devel -y
69
+
70
# install development dependencies (for testing)
71
RUN python3 -m pip install -e tests/vllm_test_utils
72
ENTRYPOINT ["python3", "-m", "vllm.entrypoints.openai.api_server"]
docs/getting_started/installation/gpu/xpu.inc.md
+9
-10
@@ -3,13 +3,16 @@
3
vLLM initially supports basic model inference and serving on Intel GPU platform.
4
5
!!! warning
6
- There are no pre-built wheels or images for this device, so you must build vLLM from source.
6
+ There are no pre-built wheels for this device, so you need build vLLM from source. Or you can use pre-built images which are based on vLLM released versions.
7
8
# --8<-- [end:installation]
9
# --8<-- [start:requirements]
10
11
- Supported Hardware: Intel Data Center GPU, Intel ARC GPU
12
-- OneAPI requirements: oneAPI 2025.0
12
+- OneAPI requirements: oneAPI 2025.1
13
+- Python: 3.12
14
+!!! warning
15
+ The provided IPEX whl is Python3.12 specific so this version is a MUST.
16
17
# --8<-- [end:requirements]
18
# --8<-- [start:set-up-using-python]
@@ -24,7 +27,7 @@ Currently, there are no pre-built XPU wheels.
27
# --8<-- [end:pre-built-wheels]
28
# --8<-- [start:build-wheel-from-source]
29
27
-- First, install required [driver](https://dgpu-docs.intel.com/driver/installation.html#installing-gpu-drivers) and [Intel OneAPI](https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit.html) 2025.0 or later.
30
+- First, install required [driver](https://dgpu-docs.intel.com/driver/installation.html#installing-gpu-drivers) and [Intel OneAPI](https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit.html) 2025.1 or later.
31
- Second, install Python packages for vLLM XPU backend building:
32
33
```bash
@@ -40,14 +43,10 @@ pip install -v -r requirements/xpu.txt
43
VLLM_TARGET_DEVICE=xpu python setup.py install
44
```
45
43
-!!! note
44
- - FP16 is the default data type in the current XPU backend. The BF16 data
45
- type is supported on Intel Data Center GPU, not supported on Intel Arc GPU yet.
46
-
46
# --8<-- [end:build-wheel-from-source]
47
# --8<-- [start:pre-built-images]
48
50
-Currently, there are no pre-built XPU images.
49
+Currently, we release prebuilt XPU images at docker [hub](https://hub.docker.com/r/intel/vllm/tags) based on vLLM released version. For more information, please refer release [note](https://github.com/intel/ai-containers/blob/main/vllm).
50
51
# --8<-- [end:pre-built-images]
52
# --8<-- [start:build-image-from-source]
@@ -65,14 +64,14 @@ docker run -it \
64
# --8<-- [end:build-image-from-source]
65
# --8<-- [start:supported-features]
66
68
-XPU platform supports **tensor parallel** inference/serving and also supports **pipeline parallel** as a beta feature for online serving. We require Ray as the distributed runtime backend. For example, a reference execution like following:
67
+XPU platform supports **tensor parallel** inference/serving and also supports **pipeline parallel** as a beta feature for online serving. For **pipeline parallel**, we support it on single node with mp as the backend. For example, a reference execution like following:
68
69
```bash
70
python -m vllm.entrypoints.openai.api_server \
71
--model=facebook/opt-13b \
72
--dtype=bfloat16 \
73
--max_model_len=1024 \
75
- --distributed-executor-backend=ray \
74
+ --distributed-executor-backend=mp \
75
--pipeline-parallel-size=2 \
76
-tp=8
77
```
requirements/xpu.txt
+3
-4
@@ -11,10 +11,9 @@ jinja2>=3.1.6
11
datasets # for benchmark scripts
12
numba == 0.60.0 # v0.61 doesn't support Python 3.9. Required for N-gram speculative decoding
13
nixl==0.3.0 # for PD disaggregation
14
---extra-index-url=https://download.pytorch.org/whl/xpu
14
torch==2.8.0+xpu
15
torchaudio
16
torchvision
18
-pytorch-triton-xpu
19
---extra-index-url=https://pytorch-extension.intel.com/release-whl/stable/xpu/us/
20
-intel-extension-for-pytorch==2.8.10+xpu
17
+--extra-index-url=https://download.pytorch.org/whl/xpu
18
+
19
+intel-extension-for-pytorch @ https://intel-extension-for-pytorch.s3.us-east-1.amazonaws.com/ipex_dev/xpu/intel_extension_for_pytorch-2.8.10.post0%2Bxpu-cp312-cp312-linux_x86_64.whl