A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 05:45 UTC
main
[bugfix] fixed parameter “n” when set parameter “bestof” > 1 (#10854)
jianzheng committed almost 2 years ago
[ci/build] Job to build and push release image (#10877)
Kevin H. Luu committed almost 2 years ago
[ci/build] Change queue name for Release jobs (#10875)
Kevin H. Luu committed almost 2 years ago
Drop ROCm load format check (#10767)
wangxiyuan committed almost 2 years ago
[CI/Build] Replace mean with torch.all in test_pynccl.py (#10876)
Tyler Michael Smith committed almost 2 years ago
[Benchmark] Benchmark structured output with datasets (#10557)
Chendi.Xue committed almost 2 years ago
[Build][Bugfix] Using the correct type hint (#10866)
Gregory Shtrasberg committed almost 2 years ago
[Frontend] correctly record prefill and decode time metrics (#10853)
tomeras91 committed almost 2 years ago
[Bugfix] Only require XGrammar on x86 (#10865)
Michael Goin committed almost 2 years ago
[MISC][XPU] quick fix for XPU CI (#10859)
Yan Ma committed almost 2 years ago
[V1] VLM - Run the mm_mapper preprocessor in the frontend process (#10640)
Alexander Matveev committed almost 2 years ago
[Speculative Decoding] Move indices to device before filtering output (#10850)
Yang Zheng committed almost 2 years ago
[Core][Performance] Add XGrammar support for guided decoding and set it as default (#10785)
Aaron Pham committed almost 2 years ago
[Misc] Remove deprecated names (#10817)
Cyrus Leung committed almost 2 years ago
[Doc] Add github links for source code references (#10672)
Russell Bryant committed almost 2 years ago
[torch.compile] remove compilation_context and simplify code (#10838)
youkaichao committed almost 2 years ago
[core][distributed] add pynccl broadcast (#10843)
youkaichao committed almost 2 years ago
[Bugfix] Fix QKVParallelLinearWithShardedLora bias bug (#10844)
Jee Jee Li committed almost 2 years ago
[Model] support bitsandbytes quantization with minicpm model (#10842)
zixuanzhang226 committed almost 2 years ago
[Bugfix] Prevent benchmark_throughput.py from using duplicated random prompts (#10753)
Michael Goin committed almost 2 years ago