A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[MM] Allow skipping memory profiling for multimodal models. (#22950)
Roger Wang committed over 1 year ago
[Bugfix] fix cuda 12.6 and 11.8 build (#22952)
Jinzhen Lin committed over 1 year ago
[Bugfix] Unquote file uri before reading image (#22912)
Sayandip Dutta committed over 1 year ago
[V1] - Split Prefill and Decode for Mamba1 models (#22653)
amirai21 committed over 1 year ago
[CI] Pooling models mteb test uses enforce_eager (#22878)
wang.yuqi committed over 1 year ago
[P/D]Provide bucket algorithm rate limiter for proxy_server (#22643)
frankie committed over 1 year ago
Revert "[ROCm][AITER] Support AITER Rope ops in RotaryEmbedding Module." (#22956)
TJian committed over 1 year ago
[Mamba] - refactor: Renamed mamba_attn to mamba2_attn (#22818)
Asaf Joseph Gardin committed over 1 year ago
[Feature] Full Cuda Graph Support for Cutlass MLA and 6% E2E Throughput Improvement (#22763)
Wentao Ye committed over 1 year ago
refactor: Change scaling factors calculation for flashinfer FusedMoE (#22812)
amirkl94 committed over 1 year ago
[CI Perf] Prune tests in `tests/kernels/attention/` (#22936)
Michael Goin committed over 1 year ago
[CI Perf] Prune tests in `tests/kernels/moe/` (#22939)
Michael Goin committed over 1 year ago
[CI Perf] Prune tests in `tests/kernels/quantization/` (#22942)
Michael Goin committed over 1 year ago
[BugFix] Fix port lookup in internal DP LB tests (#22252)
Nick Hill committed over 1 year ago
[BugFix][KVConn] Fix use of `get_required_kvcache_layout` (#22734)
Nick Hill committed over 1 year ago
Revert "[Kernel] Add cuda kernel for gpt_oss activation" (#22948)
Simon Mo committed over 1 year ago
[Kernel] Add cuda kernel for gpt_oss activation (#22538)
Jee Jee Li committed over 1 year ago
[Bugfix] use flash attn on sm90 (#22933)
Yongye Zhu committed over 1 year ago
[BugFix] Fix initial DP request load imbalance (#22910)
Nick Hill committed over 1 year ago
[Core] Return final response for aborted requests from `AsyncLLM.generate` (#22283)
Nick Hill committed over 1 year ago