@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 932c6b7461
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 15, 23:30 UTC

[V1] LM Eval With Streaming Integration Tests (#11590)

Robert Shaw committed Feb 7, 2025 at 18:07 UTC 932c6b74616d25199e87c96707e8cfea3ab045c0
1 file changed +3
.buildkite/test-pipeline.yaml
+3
@@ -195,6 +195,9 @@ steps:
195 # TODO: accuracy does not match, whether setting
196 # VLLM_USE_FLASHINFER_SAMPLER or not on H100.
197 - VLLM_USE_V1=1 pytest -v -s v1/e2e
198 + # Integration test for streaming correctness (requires special branch).
199 + - pip install -U git+https://github.com/robertgshaw2-neuralmagic/lm-evaluation-harness.git@streaming-api
200 + - pytest -v -s entrypoints/openai/test_accuracy.py::test_lm_eval_accuracy_v1_engine
201
202 - label: Examples Test # 25min
203 working_dir: "/vllm-workspace/examples"