A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 9, 10:00 UTC
main
[ModelRunner V2] Misc code simplification and cleanup (#33266)
Nick Hill committed 9 months ago
[7/N][Attention][Docs] Add documentation for attention backends (#32477)
Matthew Bonanni committed 9 months ago
[UX] Enable nested configs in config yaml files (#33193)
Michael Goin committed 9 months ago
[Bugfix] Add missing encoder only guard for do_kv_cache_update (#33269)
Gregory Shtrasberg committed 9 months ago
[ez] Remove checks for torch version <= 2.8 (#33209)
Angela Yi committed 9 months ago
Use aiter triton fused_add_rmsnorm_pad for gpt-oss (#30976)
Rohan Potdar committed 9 months ago
Wentao Ye committed 9 months ago
[CI] Change GPU key to device key for B200 test (#33275)
Kevin H. Luu committed 9 months ago
[Perf] Optimize `moe_permute` for CUTLASS FP8 (#32892)
Wentao Ye committed 9 months ago
[CI] Whisper tests `enforce_eager=False` (#33098)
Nicolò Lucchesi committed 9 months ago
cwazai committed 9 months ago
[Benchmark] Add startup benchmarking to buildkite run (#33183)
Bin Bao committed 9 months ago
[Quantization][Deprecation] Remove Marlin 24 (#32688)
Robert Shaw committed 9 months ago
[Misc] Provide a DeepSeek ReasoningParser with thinking enabled by default (#33221)
Chauncey committed 9 months ago
Revert "Enable Cross layers KV cache layout at NIXL Connector (#30207)" (#33241)
Or Ozeri committed 9 months ago
[Quantization][Deprecation] Remove BitBlas (#32683)
Robert Shaw committed 9 months ago
[CI] Update job dependency syntax for Intel and AMD jobs (#33240)
Kevin H. Luu committed 10 months ago
[CI] Update job dependency for hardware and CPU jobs (#33237)
Kevin H. Luu committed 10 months ago
[CI] Enable mypy import following for `vllm/compilation` (#33199)
Harry Mellor committed 10 months ago
[XPU]disable test_acceptance_length UT (#33226)
Yan Ma committed 10 months ago