A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 4, 01:15 UTC
main
[Attention][Cleanup] Remove tree attention (#42121)
Lucas Wilkinson committed 6 months ago
[XPU] remove q/k/v force contiguous for flash_attn (#40356)
Yan Ma committed 6 months ago
Sumanth R Hegde committed 6 months ago
Use `pre-commit / pre-run-check` to gate docs build too (#42053)
Harry Mellor committed 6 months ago
[Bugfix][KV Transfer] Reject NixlConnector + expandable_segments:True (#41237)
Roy Wang committed 6 months ago
[Bugfix] Fix FlashInfer CUTLASS MXFP4-MXFP8 MoE by restoring swizzled scale (#42089)
Michael Goin committed 6 months ago
[CUDA][CUTLASS] Enable cutlass scaled mm for non-compatible sizes (#41868)
John Calderon committed 6 months ago
use HIP_VERSION variables to guard against duplicate atomicAdd definitions (#41802)
pmaybank committed 6 months ago
[CI] Narrow misc.yaml source dependencies (#42059)
Kevin H. Luu committed 6 months ago
[CI] Narrow engine.yaml source dependencies (#42055)
Kevin H. Luu committed 6 months ago
[CI] Narrow Platform Tests (CUDA) source dependencies (#42054)
Kevin H. Luu committed 6 months ago
[CI] Narrow pytorch.yaml compile job source dependencies (#42057)
Kevin H. Luu committed 6 months ago
[Bugfix][Gemma4] Fix infinite loop and array boundary issues in tool parser (#41991)
David Oy committed 6 months ago
[ROCm][Perf] Add Fused Shared Expert (FSE) support for Qwen3-Next (#39280)
Nico Holmberg committed 6 months ago
[CI][Elastic EP] Fix Elastic EP Scaling Test Failure (#41792)
haosdent committed 6 months ago
[Models][Gemma3/Gemma4] Support hidden_act variants in gated MLP (#40588)
Hiroaki Mikami committed 6 months ago
[Docs] Fix OpenAI batch model argument examples (#42066)
Ethan Feng committed 6 months ago
[CI][XPU] Skip fork-dependent logits processor test (#42013)
liuzhenwei committed 6 months ago
[CI][Examples][RLHF] Disable async scheduling in rlhf_async_new_apis (#42042)
haosdent committed 6 months ago
Make docs environment deterministic (#41926)
Harry Mellor committed 6 months ago