A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 01:00 UTC
main
[Perf] Add swap_ab to SM90 FP8 non-block CUTLASS moe grouped gemm (#20911)
shixianc committed over 1 year ago
[Core] FlashInfer CUTLASS fused MoE backend (NVFP4) (#20037)
Shu Wang committed over 1 year ago
[Doc] Add inplace weights loading example (#19640)
22quinn committed over 1 year ago
[Attention] Make local attention backend agnostic (#21093)
Lucas Wilkinson committed over 1 year ago
[Docs] Update supported models documentation with missing models (#20844)
Lucia Fang committed over 1 year ago
[Docs] Add minimal demo of Ray Data API usage (#21080)
Ricardo Decal committed over 1 year ago
elvischenv committed over 1 year ago
[Log] Debugging Log with more Information (#20770)
Wentao Ye committed over 1 year ago
[V0 deprecation] Remove V0 HPU backend (#21131)
Woosuk Kwon committed over 1 year ago
On environments where numa cannot be detected we get 0 (#21115)
Eric Curtin committed over 1 year ago
[Misc] Qwen MoE model supports LoRA (#20932)
Jee Jee Li committed over 1 year ago
[Model] Update pooling model interface (#21058)
Cyrus Leung committed over 1 year ago
[Performance] Performance improvements in non-blockwise fp8 CUTLASS MoE (#20762)
ElizaWszola committed over 1 year ago
[Docs] Move code block out of admonition now that it's short (#21118)
Harry Mellor committed over 1 year ago
[Misc] Avoid unnecessary import (#21106)
wangxiyuan committed over 1 year ago
[Docs] Improve docstring formatting for `FusedMoEParallelConfig.make` (#21117)
Harry Mellor committed over 1 year ago
[VLM] Add Nemotron-Nano-VL-8B-V1 support (#20349)
kYLe committed over 1 year ago
[Model] Add ToolParser and MoE Config for Hunyuan A13B (#20820)
Asher committed over 1 year ago
[Kernel] DeepGemm MoE : Integrate triton permute / unpermute kernels (#20903)
Varun Sundar Rabindranath committed over 1 year ago
[Bugfix]: Fix final_res_batch list index out of range error (#21055)
Chauncey committed over 1 year ago