A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 09:30 UTC
main
[CI/Build] Adopt Mergify for auto-labeling PRs (#9259)
Russell Bryant committed about 2 years ago
[Bugfix] Fix ray instance detect issue (#9439)
Yan Ma committed about 2 years ago
Robert Shaw committed about 2 years ago
[torch.compile] support moe models (#9632)
youkaichao committed about 2 years ago
[Hardware][ROCM] using current_platform.is_rocm (#9642)
wangshuai09 committed about 2 years ago
[Bugfix] Fix load config when using bools (#9533)
madt2709 committed about 2 years ago
Fix cache management in "Close inactive issues and PRs" actions workflow (#9734)
Harry Mellor committed about 2 years ago
[Misc] Upgrade to pytorch 2.5 (#9588)
bnellnm committed about 2 years ago
[core] cudagraph output with tensor weak reference (#9724)
youkaichao committed about 2 years ago
[Misc] SpecDecodeWorker supports profiling (#9719)
科英 committed about 2 years ago
[Model] Add classification Task with Qwen2ForSequenceClassification (#9704)
kakao-kevin-us committed about 2 years ago
[Frontend] Bad words sampling parameter (#9717)
Vasiliy Alekseev committed about 2 years ago
Fix: MI100 Support By Bypassing Custom Paged Attention (#9560)
ErkinSagiroglu committed about 2 years ago
[Hardware][openvino] is_openvino --> current_platform.is_openvino (#9716)
Mengqing Cao committed about 2 years ago
[Bugfix] Steaming continuous_usage_stats default to False (#9709)
Sam Stoelinga committed about 2 years ago
[Bugfix] Fix crash with llama 3.2 vision models and guided decoding (#9631)
Travis Johnson committed about 2 years ago
[Doc] Improve quickstart documentation (#9256)
Rafael Vasquez committed about 2 years ago
[Bugfix] Fix compressed_tensors_moe bad config.strategy (#9677)
Michael Goin committed about 2 years ago
[V1] Support sliding window attention (#9679)
Woosuk Kwon committed about 2 years ago
[Model] add a lora module for granite 3.0 MoE models (#9673)
Will Johnson committed about 2 years ago