A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 07:15 UTC
main
[CI/Build] remove regex from build dependencies (#18945)
Daniele committed over 1 year ago
[Bugfix][TPU] Fix tpu model runner testcase failure (#18810)
Carol Zheng committed over 1 year ago
Always-Naive committed over 1 year ago
[Bugfix][Failing Test] Fix test_vllm_port.py (#18618)
Rabi Mishra committed over 1 year ago
[Model] Use in-place adds in SigLIP (#18922)
Lukas Geiger committed over 1 year ago
Reid committed over 1 year ago
[Misc]Fix benchmarks/README.md for speculative decoding (#18897)
Rabi Mishra committed over 1 year ago
[Deprecation] Remove mean pooling default for `Qwen2EmbeddingModel` (#18913)
Cyrus Leung committed over 1 year ago
[Bugfix] Remove NVFP4 scales assertions to fix load_format=dummy (#18861)
Michael Goin committed over 1 year ago
[ROCm] Remove unnecessary assertion of max_model_len in ROCM_AITER_MLA attention backend. (#18938)
vllmellm committed over 1 year ago
[docs] fix: fix markdown syntax (#18927)
H committed over 1 year ago
[Model] Use AutoWeightsLoader for mamba2 (#18918)
iLeGend committed over 1 year ago
[Bugfix] Consistent ascii handling in tool parsers (#18883)
Chauncey committed over 1 year ago
improve the robustness of parsing vlms config in AutoRound (#18894)
Wenhua Cheng committed over 1 year ago
[TPU][CI/CD] Clean up docker for TPU tests. (#18926)
Carol Zheng committed over 1 year ago
[Misc] Update type annotation for rotary embedding `base` (#18914)
Cyrus Leung committed over 1 year ago
[Bugfix] Fix PP default fallback behavior for V1 (#18915)
Michael Goin committed over 1 year ago
[TPU] remove transpose ops in moe kernel (#18923)
Chengji Yao committed over 1 year ago
Use standalone_compile by default in torch >= 2.8.0 (#18846)
Richard Zou committed over 1 year ago
[P/D] NixlConnector DP fixes (#18903)
Will Eaton committed over 1 year ago