A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 14:45 UTC
[Bugfix] fix gpu docker image mis benchmarks dir (#16628)
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
rongfu.leng committed
Apr 16, 2025 at 12:21 UTC
96bb8aa68b2a155e2c4dc574afcb8d7e3d8bf9a1
1 file changed
+1
docker/Dockerfile
+1
@@ -240,6 +240,7 @@ if [ "$TARGETPLATFORM" != "linux/arm64" ]; then \
240
uv pip install --system https://github.com/flashinfer-ai/flashinfer/releases/download/v0.2.1.post2/flashinfer_python-0.2.1.post2+cu124torch2.6-cp38-abi3-linux_x86_64.whl ; \
241
fi
242
COPY examples examples
243
+COPY benchmarks benchmarks
244
245
# Although we build Flashinfer with AOT mode, there's still
246
# some issues w.r.t. JIT compilation. Therefore we need to