A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 01:45 UTC
main
[docs] Fix typo in examples filename openi -> openai (#4864)
Alex Wu committed over 2 years ago
[Kernel] add bfloat16 support for gptq marlin kernel (#4788)
Jinzhen Lin committed over 2 years ago
Add marlin unit tests and marlin benchmark script (#4815)
alexm-nm committed over 2 years ago
[Speculative decoding][Re-take] Enable TP>1 speculative decoding (#4840)
Cody Yu committed over 2 years ago
[Core] Implement sharded state loader (#4690)
Aurick Qiao committed over 2 years ago
[Frontend] Support OpenAI batch file format (#4794)
Alex Wu committed over 2 years ago
[Frontend] Re-enable custom roles in Chat Completions API (#4758)
Cyrus Leung committed over 2 years ago
[Doc] Highlight the fourth meetup in the README (#4842)
Zhuohan Li committed over 2 years ago
[Bugfix] Properly set distributed_executor_backend in ParallelConfig (#4816)
zifeitong committed over 2 years ago
[CI/Build] Further decouple HuggingFace implementation from ours during tests (#4166)
Cyrus Leung committed over 2 years ago
SangBin Cho committed over 2 years ago
Revert "[Kernel] Use flash-attn for decoding (#3648)" (#4820)
SangBin Cho committed over 2 years ago
Add 4th meetup announcement to readme (#4817)
Simon Mo committed over 2 years ago
[Core] Add MultiprocessingGPUExecutor (#4539)
Nick Hill committed over 2 years ago
[Bugfix][Doc] Fix CI failure in docs (#4804)
Cyrus Leung committed over 2 years ago
Kuntai Du committed over 2 years ago
[Doc] Add meetups to the doc (#4798)
Zhuohan Li committed over 2 years ago
[Doc] Add API reference for offline inference (#4710)
Cyrus Leung committed over 2 years ago
[Doc] Shorten README by removing supported model list (#4796)
Zhuohan Li committed over 2 years ago
[Bugfix] Fix dynamic FP8 quantization for Mixtral (#4793)
Philipp Moritz committed over 2 years ago