A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 11:00 UTC
main
[Misc] Fix ScalarType float4 naming (#17690)
Lucas Wilkinson committed over 1 year ago
[Doc] Update notes for H2O-VL and Gemma3 (#17219)
Cyrus Leung committed over 1 year ago
[Bugfix] Fixed prompt length for random dataset (#17408)
Mikhail Podvitskii committed over 1 year ago
[easy] Fix logspam on PiecewiseBackend errors (#17138)
Richard Zou committed over 1 year ago
[Model] Add GraniteMoeHybrid 4.0 model (#17497)
Stan Wozniak committed over 1 year ago
Update nm to rht in doc links + refine fp8 doc (#17678)
Michael Goin committed over 1 year ago
[Bugfix] LoRA - Retire unused maxnreg LoRA kernel argument (#17677)
Varun Sundar Rabindranath committed over 1 year ago
[TPU][V1] Add support for top-logprobs (#17072)
Nicolò Lucchesi committed over 1 year ago
[TPU] Enable gemma3-27b with TP>1 on multi-chips. (#17335)
XiongfeiWei committed over 1 year ago
[BugFix] Increase timeout for startup failure test (#17642)
Nick Hill committed over 1 year ago
[Benchmarks] Remove invalid option under V1 engine (#17651)
Russell Bryant committed over 1 year ago
Use git-path commit in hook (#17616)
Thomas J. Fan committed over 1 year ago
[Doc] Fix broken cuda installation doc rendering (#17654)
Isotr0py committed over 1 year ago
[Kernel] some optimizations for dense marlin and moe marlin (#16850)
Jinzhen Lin committed over 1 year ago
[Build/CI] Upgrade CUTLASS to 3.9.2 (#17641)
Tyler Michael Smith committed over 1 year ago
[Bugfix] fix KeyError on top logprobs are special tokens (#17637)
Chauncey committed over 1 year ago
[Bugfix][Easy] Fix whitespace in shm_broadcast.py logging (#17635)
Tyler Michael Smith committed over 1 year ago
[Bugfix] Prioritize dtype in root config before checking text config (#17629)
Cyrus Leung committed over 1 year ago
Add full API docs and improve the UX of navigating them (#17485)
Harry Mellor committed over 1 year ago
[Misc] V0 fallback for `--enable-prompt-embeds` (#17615)
Cyrus Leung committed over 1 year ago