A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 01:00 UTC
main
[BugFix][CPU] Fix `TorchSDPABackendImpl` doesn't have `use_irope` (#21200)
Lucas Wilkinson committed over 1 year ago
[Kernel][Performance] Tweak MoE Batched silu_mul_fp8_quant_deep_gemm kernel (#21193)
Varun Sundar Rabindranath committed over 1 year ago
[V0 Deprecation] Remove V0 Spec Decode workers (#21152)
Woosuk Kwon committed over 1 year ago
[Bugfix][Model] Fix LoRA for Mistral-Small-3.1-24B-Instruct-2503 (#21183)
Varun Sundar Rabindranath committed over 1 year ago
[Core] Support Local Chunked Attention for Hybrid KV Cache (#19351)
Lucia Fang committed over 1 year ago
[Quantization] Enable BNB support for more MoE models (#21100)
Jee Jee Li committed over 1 year ago
Elastic Expert Parallel Initial Support (#20775)
Rui Qiao committed over 1 year ago
[Bugfix] Voxtral on Blackwell GPUs (RTX 50 series) (#21077)
hax0r31337 committed over 1 year ago
[Core] Avoid KVCacheBlock.__eq__ invocations in FreeKVCacheBlockQueue (#21005)
JialinOuyang-Meta committed over 1 year ago
[Kernel] Apply torch.Tag.needs_fixed_stride_order only for torch==2.6.0 (#19346)
Richard Zou committed over 1 year ago
[CI] Update CODEOWNERS for vllm/compilation (#21185)
Richard Zou committed over 1 year ago
Let GraniteMoeAttention use YaRN (#21174)
Thomas Parnell committed over 1 year ago
[Core] Set pooling params based on task and model (#21128)
Cyrus Leung committed over 1 year ago
[Bugfix] Allocate less memory in non-batched CUTLASS MoE (#21121)
ElizaWszola committed over 1 year ago
[Doc] Fix typo in model name (#21178)
Cyrus Leung committed over 1 year ago
wang.yuqi committed over 1 year ago
[Model] Re-add the implicit conversion feature for as_seq_cls_model (#21103)
wang.yuqi committed over 1 year ago
[Misc] Make MM embedding merge interface explicit in model runner (#21147)
Roger Wang committed over 1 year ago
[benchmark] Sending request strictly follows the random intervals (#21108)
Jialin Ouyang committed over 1 year ago
[Misc] Do not print async output warning for v1 (#21151)
Woosuk Kwon committed over 1 year ago