A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 13:30 UTC
main
Bump Transformers to 4.51.3 (#17116)
Harry Mellor committed over 1 year ago
[Bugfix] Fix Mistral ChatCompletionRequest Body Exception (#16769)
Jasmond L committed over 1 year ago
[Bugfix] Fix mistral model tests (#17181)
Cyrus Leung committed over 1 year ago
[Doc] Move todo out of beam search docstring (#17183)
Alex Brooks committed over 1 year ago
[Doc] Add two links to disagg_prefill.md (#17168)
Michael Yao committed over 1 year ago
Only turn on FastIncrementalDetokenizer when tokenizers >= 0.21.1 (#17158)
Lu Fang committed over 1 year ago
[Doc] Add headings to improve gptqmodel.md (#17164)
Michael Yao committed over 1 year ago
rasmith committed over 1 year ago
[Bugfix] remove fallback in guided_json (int range, patterns) (#16725)
Sangyeon Cho committed over 1 year ago
yexin(叶鑫) committed over 1 year ago
[Misc] Benchmark Serving Script Support Appending Results (#17028)
Lucas Wilkinson committed over 1 year ago
[Bugfix][Misc] Use TritonPlaceholderModule to defensively import triton (#15099)
Mengqing Cao committed over 1 year ago
[Misc] Clean up redundant code in uniproc_executor.py (#16762)
Lifu Huang committed over 1 year ago
Move missed `SchedulerConfig` args into scheduler config group in `EngineArgs` (#17131)
Harry Mellor committed over 1 year ago
[Docs] Fix True->true in supported_models.md (#17141)
Michael Goin committed over 1 year ago
[Doc] V1 : Update LoRA status (#17133)
Varun Sundar Rabindranath committed over 1 year ago
fix float16 support for kimi-vl (#17156)
Zaida Zhou committed over 1 year ago
[Attention] FA3 decode perf improvement - single mma warp group support for head dim 128 (#16864)
Lucas Wilkinson committed over 1 year ago
[FEAT] [ROCm]: AITER Fused MOE V1 Support (#16752)
vllmellm committed over 1 year ago
Use custom address for listening socket (#15988)
jglaser committed over 1 year ago