@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 8b798eec75
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 01:45 UTC

[CI/Build][Bugfix] VLLM_USE_PRECOMPILED should skip compilation (#4534)

Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>

Travis Johnson committed May 1, 2024 at 12:01 UTC 8b798eec75cde6eb6fe65b5d673dd9bd4eaef799
1 file changed +1
setup.py
+1
@@ -378,6 +378,7 @@ package_data = {
378 "vllm": ["py.typed", "model_executor/layers/fused_moe/configs/*.json"]
379 }
380 if os.environ.get("VLLM_USE_PRECOMPILED"):
381 + ext_modules = []
382 package_data["vllm"].append("*.so")
383
384 setup(