A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 05:30 UTC
main
Fix progress bar and allow HTTPS in `benchmark_serving.py` (#2552)
Harry Mellor committed almost 3 years ago
Junyang Lin committed almost 3 years ago
Fix https://github.com/vllm-project/vllm/issues/2540 (#2545)
Philipp Moritz committed almost 3 years ago
[Speculative decoding 2/9] Multi-step worker for draft model (#2424)
Cade Daniel committed almost 3 years ago
migrate pydantic from v1 to v2 (#2531)
Jannis Schönleber committed almost 3 years ago
[Fix] Keep `scheduler.running` as deque (#2523)
Nick Hill committed almost 3 years ago
Add `group` as an argument in broadcast ops (#2522)
Junda Chen committed almost 3 years ago
Add benchmark serving to CI (#2505)
Simon Mo committed almost 3 years ago
[Bugfix] fix load local safetensors model (#2512)
Roy committed almost 3 years ago
Simplify broadcast logic for control messages (#2501)
Zhuohan Li committed almost 3 years ago
Support OpenAI API server in `benchmark_serving.py` (#2172)
Harry Mellor committed almost 3 years ago
refactor complemention api for readability (#2499)
Simon Mo committed almost 3 years ago
[BugFix] Fix abort_seq_group (#2463)
ljss committed almost 3 years ago
Don't download both safetensor and bin files. (#2480)
Nikola Borisov committed almost 3 years ago
[Neuron] Add an option to build with neuron (#2065)
Liangfu Chen committed almost 3 years ago
fix: fix some args desc (#2487)
zspo committed almost 3 years ago
Minor fix in prefill cache example (#2494)
Jason Zhu committed almost 3 years ago
fix stablelm.py tensor-parallel-size bug (#2482)
YingchaoX committed almost 3 years ago
[Experimental] Prefix Caching Support (#1669)
shiyi.c_98 committed almost 3 years ago
OpenAI Server refactoring (#2360)
FlorianJoncour committed almost 3 years ago