@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / c5602f0baa
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 23:15 UTC

[ci] Mount buildkite agent on Docker container to upload benchmark results (#5330)

Signed-off-by: kevin <kevin@anyscale.com>

Kevin H. Luu committed Jun 10, 2024 at 09:22 UTC c5602f0baa8fa42df11853a0d422bc140cf04c9a
2 files changed +6 -3
.buildkite/run-benchmarks.sh
+3 -3
@@ -54,12 +54,12 @@ tail -n 24 benchmark_serving.txt >> benchmark_results.md # last 24 lines
54 echo '```' >> benchmark_results.md
55
56 # if the agent binary is not found, skip uploading the results, exit 0
57 -if [ ! -f /workspace/buildkite-agent ]; then
57 +if [ ! -f buildkite-agent ]; then
58 exit 0
59 fi
60
61 # upload the results to buildkite
62 -/workspace/buildkite-agent annotate --style "info" --context "benchmark-results" < benchmark_results.md
62 +buildkite-agent annotate --style "info" --context "benchmark-results" < benchmark_results.md
63
64 # exit with the exit code of the benchmarks
65 if [ $bench_latency_exit_code -ne 0 ]; then
@@ -75,4 +75,4 @@ if [ $bench_serving_exit_code -ne 0 ]; then
75 fi
76
77 rm ShareGPT_V3_unfiltered_cleaned_split.json
78 -/workspace/buildkite-agent artifact upload "*.json"
78 +buildkite-agent artifact upload "*.json"
.buildkite/test-template-aws.j2
+3
@@ -49,6 +49,9 @@ steps:
49 {% if not step.no_gpu %}
50 gpus: all
51 {% endif %}
52 + {% if step.label == "Benchmarks" %}
53 + mount-buildkite-agent: true
54 + {% endif %}
55 command: ["bash", "-c", "cd {{ (step.working_dir or default_working_dir) | safe }} && {{ step.command or (step.commands | join(' && ')) | safe }}"]
56 environment:
57 - VLLM_USAGE_SOURCE=ci-test