@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 855c8ae2c9
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 15:45 UTC

[MISC] remove engine_use_ray in benchmark_throughput.py (#8615)

Kunshang Ji committed Sep 19, 2024 at 13:33 UTC 855c8ae2c9a4085b1ebd66d9a978fb23f47f822c
1 file changed -1
benchmarks/benchmark_throughput.py
-1
@@ -191,7 +191,6 @@ async def run_vllm_async(
191 use_v2_block_manager=use_v2_block_manager,
192 disable_async_output_proc=disable_async_output_proc,
193 worker_use_ray=False,
194 - engine_use_ray=False,
194 disable_log_requests=True,
195 )
196