@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 949e49a685
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 23:15 UTC

[ci] Limit num gpus if specified for A100 (#5694)

Signed-off-by: kevin <kevin@anyscale.com>

Kevin H. Luu committed Jun 19, 2024 at 16:30 UTC 949e49a6857080e36ecd62f6e193754290c7c43c
2 files changed +2 -1
.buildkite/test-pipeline.yaml
+1
@@ -184,6 +184,7 @@ steps:
184
185 - label: Distributed Tests (A100)
186 gpu: a100
187 + num_gpus: 4
188 commands:
189 # NOTE: don't test llama model here, it seems hf implementation is buggy
190 # see https://github.com/vllm-project/vllm/pull/5689 for details
.buildkite/test-template-aws.j2
+1 -1
@@ -75,7 +75,7 @@ steps:
75 - "'cd {{ (step.working_dir or default_working_dir) | safe }} && {{ step.command or (step.commands | join(' && ')) | safe }}'"
76 resources:
77 limits:
78 - nvidia.com/gpu: 8
78 + nvidia.com/gpu: {{ step.num_gpus or 1 }}
79 volumeMounts:
80 - name: devshm
81 mountPath: /dev/shm