A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[ROCm][Build] Add support for AMD Ryzen AI MAX / AI 300 Series (#25908)
Hosang committed about 1 year ago
[BugFix][DP/EP] Fix CUTLASS MLA hang under load (#26026)
Lucas Wilkinson committed about 1 year ago
[CI] Tweaks to GPT-OSS Eval (Blackwell) for stability (#26030)
Michael Goin committed about 1 year ago
Fix test_mamba_ssm_ssd.py due to missing _query_start_loc_to_chunk_indices_offsets (#25995)
Huamin Li committed about 1 year ago
[NVIDIA] Blackwell Family (#24673)
Johnny committed about 1 year ago
[Bugfix] Apply same sampling parameters for both `n=1` and `n>1` (#26005)
Kenichi Maehashi committed about 1 year ago
[Benchmark] Finish documented v0.11.0 deprecation of --endpoint-type (#26007)
Nathan Scott committed about 1 year ago
[MISC] Fix misleading batch_size_capture_list when cuda_graph_sizes < 4 (#25829)
billishyahao committed about 1 year ago
[CI] Only capture a single CUDA graph size in CI by default (#25951)
Harry Mellor committed about 1 year ago
[Misc] Factor out common `_apply_feature_select_strategy` (#26003)
Cyrus Leung committed about 1 year ago
[BugFix][MM] Fix Nonetype error when video is cache in qwen2.5-omni-thinker (#26004)
Wenlong Wang committed about 1 year ago
[Doc] updating torch.compile doc link (#25989)
nadathurv committed about 1 year ago
[Misc]allow disable pynccl (#25421)
Lucia Fang committed about 1 year ago
Update to Transformers `v4.56.2` (#24638)
Harry Mellor committed about 1 year ago
[BugFix] Fix default kv-cache-dtype default for DeepseekV3.2 (#25988)
Lucas Wilkinson committed about 1 year ago
[Bugfix] Fix `__syncwarp` on ROCM (#25996)
Zhewen Li committed about 1 year ago
[MM] Add text-only mode for Qwen3-VL (#26000)
Roger Wang committed about 1 year ago
Fix INT8 quantization error on Blackwell GPUs (SM100+) (#25935)
Param committed about 1 year ago
[Log] Optimize Log for FP8MOE (#25709)
Wentao Ye committed about 1 year ago
Salvatore Cena committed about 1 year ago