A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[LoRA] Much faster startup when LoRA is enabled (#23777)
Andy Lo committed about 1 year ago
[Misc] enhance type hint for rearrange return value (#23519)
Ning Xie committed about 1 year ago
[Refactor] refactor freezing_value/cuda_event initialize outside try finally (#23758)
Ning Xie committed about 1 year ago
[Misc] add reorder_batch AttentionMetadataBuilder (#23798)
Ning Xie committed about 1 year ago
Add LoRA support for DeepSeek models (V2, V3, R1-0528) (#23971)
sadegh.shokatian committed about 1 year ago
[Model] Enable encoder DP for MiniCPM-V (#23948)
Jiangyun Zhu committed about 1 year ago
[UT] fix unify_kv_cache_configs when kv cache config needs sort (#23843)
Ning Xie committed about 1 year ago
[Bugfix] Fix test_lora_resolvers.py (#23984)
Jee Jee Li committed about 1 year ago
[V1] [Hybrid] Move MiniMaxLinearAttention into layers/mamba (#23831)
Thomas Parnell committed about 1 year ago
[Core] Cleanup TPU model runner for MM (#23894)
Cyrus Leung committed about 1 year ago
[CI] Fix broken compile tests due to unsupported SiluMul+Nvfp4Quant fusion (#23973)
Yong Hoon Shin committed about 1 year ago
[CI] Move testing image from remote URL to S3 (#23980)
Roger Wang committed about 1 year ago
Add routed_scaling_factor to MoE grouped topk (#23123)
Xin Yang committed about 1 year ago
[Bugfix] Fix --config arg expansion called from api_server.py (#23944)
dubejf committed about 1 year ago
[CI] Fix unavailable image remote URL (#23966)
Roger Wang committed about 1 year ago
[Misc] Make `download_weights_from_hf` more reliable (#23863)
Harry Mellor committed about 1 year ago
Revert gemma3n fast prefill changes (#23897)
Yong Hoon Shin committed about 1 year ago
[Docs] [V1] [Hybrid] Add new documentation re: contributing mamba-based models (#23824)
Thomas Parnell committed about 1 year ago
Tuned H100/H200 triton fp8 block configs for fused_qkv_a_proj (#23939)
Michael Goin committed about 1 year ago
[RL][BugFix] Fix missing tokenizer error for token-in-token-out (#23904)
22quinn committed about 1 year ago