@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / bb60326836
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 22:00 UTC

[Misc] update benchmark backend for scalellm (#6018)

zhyncs committed Jul 2, 2024 at 01:20 UTC bb6032683687b817b026a38c5a8729c2cd861246
1 file changed +1
benchmarks/backend_request_func.py
+1
@@ -422,4 +422,5 @@ ASYNC_REQUEST_FUNCS = {
422 "openai": async_request_openai_completions,
423 "openai-chat": async_request_openai_chat_completions,
424 "tensorrt-llm": async_request_trt_llm,
425 + "scalellm": async_request_openai_completions,
426 }