@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / d23679eb99
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 15:45 UTC

[Bugfix] fix docker build for xpu (#8652)

Yan Ma committed Sep 23, 2024 at 13:54 UTC d23679eb9960ad2a876b88ebd0028dbe55c3172a
3 files changed +6 -14
Dockerfile.xpu
+1 -9
@@ -10,19 +10,11 @@ RUN wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRO
10 RUN apt-get update -y \
11 && apt-get install -y curl libicu70 lsb-release git wget vim numactl python3 python3-pip ffmpeg libsm6 libxext6 libgl1
12
13 -RUN git clone https://github.com/intel/pti-gpu && \
14 - cd pti-gpu/sdk && \
15 - mkdir build && \
16 - cd build && \
17 - cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=../cmake/toolchains/icpx_toolchain.cmake -DBUILD_TESTING=OFF .. && \
18 - make -j && \
19 - cmake --install . --config Release --prefix "/usr/local"
20 -
13 COPY ./ /workspace/vllm
14
15 WORKDIR /workspace/vllm
16
25 -RUN pip install -v -r requirements-xpu.txt
17 +RUN pip install -v -r requirements-xpu.txt --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/
18
19 RUN VLLM_TARGET_DEVICE=xpu python3 setup.py install
20
docs/source/getting_started/xpu-installation.rst
+3 -3
@@ -17,8 +17,8 @@ Requirements
17 ------------
18
19 * OS: Linux
20 -* Supported Hardware: Intel Data Center GPU (Intel ARC GPU WIP)
21 -* OneAPI requirements: oneAPI 2024.1
20 +* Supported Hardware: Intel Data Center GPU, Intel ARC GPU
21 +* OneAPI requirements: oneAPI 2024.2
22
23 .. _xpu_backend_quick_start_dockerfile:
24
@@ -40,7 +40,7 @@ Quick start using Dockerfile
40 Build from source
41 -----------------
42
43 -- First, install required driver and intel OneAPI 2024.1 or later.
43 +- First, install required driver and intel OneAPI 2024.2 or later.
44
45 - Second, install Python packages for vLLM XPU backend building:
46
requirements-xpu.txt
+2 -2
@@ -3,10 +3,10 @@
3
4 setuptools < 70.0.0 # IPEX's torch have some dependency. to be removed.
5
6 +ray >= 2.9
7 +# Following pkgs retrieved from https://pytorch-extension.intel.com/release-whl/stable/xpu/us/
8 torch == 2.3.1+cxx11.abi
9 intel-extension-for-pytorch == 2.3.110+xpu
10 oneccl_bind_pt == 2.3.100+xpu
11
12 triton-xpu == 3.0.0b2
11 -
12 ---extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/