A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 22:35 UTC
main
[Frontend] Align tool_choice="required" behavior with OpenAI when tools is empty (#21052)
Sungyoon Jeong committed over 1 year ago
[Core] Avoid repeated len(block_token_ids) check in hash_request_tokens (#21781)
Zebing Lin committed over 1 year ago
[Bugfix] Disable multi-modal preprocessor cache for DP (#21896)
Cyrus Leung committed over 1 year ago
[Doc] Fix a syntax error of example code in structured_outputs.md (#22045)
Hongsheng Liu committed over 1 year ago
Update sampling_metadata.py (#21937)
Aviad Rossmann committed over 1 year ago
[Doc] Add Voxtral to Supported Models page (#22059)
Cyrus Leung committed over 1 year ago
[Model] [Quantization] Support quantization for Gemma3n (#21974)
Kyle Sayers committed over 1 year ago
[BugFix] fix: aot passes kvcache dtype information (#19750)
Mickaël Seznec committed over 1 year ago
[Misc] Automatically resolve HF processor init kwargs (#22005)
Cyrus Leung committed over 1 year ago
[Bugfix] Fix: Fix multi loras with tp >=2 and LRU cache (#20873)
Charent committed over 1 year ago
[Refactor] Remove Duplicate `per_block_cast_to_fp8`, Remove Dependencies of DeepGEMM (#21787)
Wentao Ye committed over 1 year ago
Move flashinfer-python to optional extra `vllm[flashinfer]` (#21959)
Michael Goin committed over 1 year ago
Add DeepGEMM to Dockerfile in vllm-base image (#21533)
Matthew Bonanni committed over 1 year ago
[Feature] Add Flashinfer MoE Support for Compressed Tensor NVFP4 (#21639)
Wentao Ye committed over 1 year ago
Add FlashInfer allreduce RMSNorm Quant fusion (#21069)
Ilya Markov committed over 1 year ago
[Bugfix] Fix MTP weight loading (#21941)
Benjamin Chislett committed over 1 year ago
[Misc] Add unit tests for chunked local attention (#21692)
Yong Hoon Shin committed over 1 year ago
[Meta] Official Eagle mm support, first enablement on llama4 (#20788)
zhiweiz committed over 1 year ago
Update torch_xla pin to 20250730 (#21956)
XiongfeiWei committed over 1 year ago
Removing amdproduction Tests (#22027)
Alexei-V-Ivanov-AMD committed over 1 year ago