A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 23:15 UTC
[CI][Hardware][Intel GPU] add Intel GPU(XPU) ci pipeline (#5616)
Kunshang Ji committed
Jun 22, 2024 at 08:09 UTC
cf90ae01237018f70573f69c599d26648ff7740b
2 files changed
+9
-3
.buildkite/test-template-aws.j2
+8
-2
@@ -42,12 +42,18 @@ steps:
42
command: bash .buildkite/run-neuron-test.sh
43
soft_fail: false
44
45
- - label: "Intel Test"
45
+ - label: "Intel CPU Test"
46
depends_on: ~
47
agents:
48
- queue: intel
48
+ queue: intel-cpu
49
command: bash .buildkite/run-cpu-test.sh
50
51
+ - label: "Intel GPU Test"
52
+ depends_on: ~
53
+ agents:
54
+ queue: intel-gpu
55
+ command: bash .buildkite/run-xpu-test.sh
56
+
57
{% for step in steps %}
58
{% if step.gpu == "a100" %}
59
- label: "{{ step.label }}"
README.md
+1
-1
@@ -59,7 +59,7 @@ vLLM is flexible and easy to use with:
59
- Tensor parallelism support for distributed inference
60
- Streaming outputs
61
- OpenAI-compatible API server
62
-- Support NVIDIA GPUs, AMD GPUs, and Intel CPUs
62
+- Support NVIDIA GPUs, AMD GPUs, Intel CPUs and GPUs
63
- (Experimental) Prefix caching support
64
- (Experimental) Multi-lora support
65