A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 00:45 UTC
main
Disable chunked prefill and/or prefix caching when MLA is enabled (#12642)
Simon Mo committed almost 2 years ago
[CI/Build] Add label automation for structured-output, speculative-decoding, v1 (#12280)
Russell Bryant committed almost 2 years ago
[Attention] Deepseek v3 MLA support with FP8 compute (#12601)
Lucas Wilkinson committed almost 2 years ago
Fix: Respect `sparsity_config.ignore` in Cutlass Integration (#12517)
Rahul Tuli committed almost 2 years ago
[Bugfix/CI] Fixup benchmark_moe.py (#12562)
Tyler Michael Smith committed almost 2 years ago
[ci] Upgrade transformers to 4.48.2 in CI dependencies (#12599)
Kevin H. Luu committed almost 2 years ago
Fix target matching for fused layers with compressed-tensors (#12617)
Eldar Kurtic committed almost 2 years ago
[BugFix] fix wrong output when using lora and num_scheduler_steps=8 (#11161)
fade_away committed almost 2 years ago
[V1] Bugfix: Validate Model Input Length (#12600)
Robert Shaw committed almost 2 years ago
[Doc] int4 w4a16 example (#12585)
Brian Dellabetta committed almost 2 years ago
[Doc] Improve installation signposting (#12575)
Harry Mellor committed almost 2 years ago
[Feature] Fix guided decoding blocking bitmask memcpy (#12563)
Ryan Nguyen committed almost 2 years ago
[Kernel][Quantization] Integrate block-quantized CUTLASS kernels for DeepSeekV3 (#12587)
Tyler Michael Smith committed almost 2 years ago
[Bugfix] Revert MoE Triton Config Default (#12629)
Robert Shaw committed almost 2 years ago
[release] Add input step to ask for Release version (#12631)
Kevin H. Luu committed almost 2 years ago
[v1][Bugfix] Add extra_keys to block_hash for prefix caching (#12603)
Chen Zhang committed almost 2 years ago
[Docs][V1] Prefix caching design (#12598)
Cody Yu committed almost 2 years ago
[Git] Automatically sign-off commits (#12595)
Cody Yu committed almost 2 years ago
[BugFix] Fix Torch.Compile For DeepSeek (#12594)
Robert Shaw committed almost 2 years ago
Harry Mellor committed almost 2 years ago