@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 1c046447a6
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 15:45 UTC

[CI/Build][Bugfix][Doc][ROCm] CI fix and doc update after ROCm 6.2 upgrade (#8777)

Hongxia Yang committed Sep 25, 2024 at 10:26 UTC 1c046447a6d1ac3c99b9f453796f0d355d673deb
3 files changed +16 -3
.buildkite/test-pipeline.yaml
+4 -1
@@ -90,8 +90,11 @@ steps:
90 commands:
91 - pip install -e ./plugins/vllm_add_dummy_model
92 - pip install git+https://github.com/EleutherAI/lm-evaluation-harness.git@a4987bba6e9e9b3f22bd3a6c1ecf0abd04fd5622#egg=lm_eval[api]
93 - - pytest -v -s entrypoints/llm --ignore=entrypoints/llm/test_lazy_outlines.py
93 + - pytest -v -s entrypoints/llm --ignore=entrypoints/llm/test_lazy_outlines.py --ignore=entrypoints/llm/test_generate.py --ignore=entrypoints/llm/test_generate_multiple_loras.py --ignore=entrypoints/llm/test_guided_generate.py
94 - pytest -v -s entrypoints/llm/test_lazy_outlines.py # it needs a clean process
95 + - pytest -v -s entrypoints/llm/test_generate.py # it needs a clean process
96 + - pytest -v -s entrypoints/llm/test_generate_multiple_loras.py # it needs a clean process
97 + - pytest -v -s entrypoints/llm/test_guided_generate.py # it needs a clean process
98 - pytest -v -s entrypoints/openai
99 - pytest -v -s entrypoints/test_chat_utils.py
100 - pytest -v -s entrypoints/offline_mode # Needs to avoid interference with other tests
Dockerfile.rocm
+1 -1
@@ -120,7 +120,7 @@ COPY . .
120
121 # Package upgrades for useful functionality or to avoid dependency issues
122 RUN --mount=type=cache,target=/root/.cache/pip \
123 - python3 -m pip install --upgrade numba scipy huggingface-hub[cli]
123 + python3 -m pip install --upgrade numba scipy huggingface-hub[cli] pytest-shard
124
125
126 # Workaround for ray >= 2.10.0
docs/source/getting_started/amd-installation.rst
+11 -1
@@ -28,6 +28,16 @@ Option 1: Build from source with docker (recommended)
28 You can build and install vLLM from source.
29
30 First, build a docker image from `Dockerfile.rocm <https://github.com/vllm-project/vllm/blob/main/Dockerfile.rocm>`_ and launch a docker container from the image.
31 +It is important that the user kicks off the docker build using buildkit. Either the user put DOCKER_BUILDKIT=1 as environment variable when calling docker build command, or the user needs to setup buildkit in the docker daemon configuration /etc/docker/daemon.json as follows and restart the daemon:
32 +
33 +.. code-block:: console
34 +
35 + {
36 + "features": {
37 + "buildkit": true
38 + }
39 + }
40 +
41
42 `Dockerfile.rocm <https://github.com/vllm-project/vllm/blob/main/Dockerfile.rocm>`_ uses ROCm 6.2 by default, but also supports ROCm 5.7, 6.0 and 6.1 in older vLLM branches.
43 It provides flexibility to customize the build of docker image using the following arguments:
@@ -152,7 +162,7 @@ Note to get your gfx architecture, run `rocminfo |grep gfx`.
162 $ python3 setup.py develop
163
164
155 - This may take 5-10 minutes. Currently, `pip install .`` does not work for ROCm installation
165 + This may take 5-10 minutes. Currently, :code:`pip install .` does not work for ROCm installation.
166
167
168 .. tip::