A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 06:00 UTC
main
[Bugfix][V1] Allow manual FlashAttention for Blackwell (#19492)
Michael Goin committed over 1 year ago
[Security] Prevent new imports of (cloud)pickle (#18018)
Russell Bryant committed over 1 year ago
niu_he committed over 1 year ago
[AMD][Kernel][BugFix] fix test_rocm_compressed_tensors_w8a8 for rocm (#19509)
rasmith committed over 1 year ago
[BugFix] Handle missing sep_token for Qwen3-Reranker in Score API (#19522)
wonjun Jang committed over 1 year ago
[BugFix] Work-around incremental detokenization edge case error (#19449)
Nick Hill committed over 1 year ago
[Frontend] Improve error message in tool_choice validation (#19239)
22quinn committed over 1 year ago
Add Triton Fused MoE kernel config for E=16 on B200 (#19518)
Brayden Zhong committed over 1 year ago
[BugFix] Force registration of w8a8_block_fp8_matmul_deepgemm via lazy import (#19514)
Varun Sundar Rabindranath committed over 1 year ago
[CI] change spell checker from codespell to typos (#18711)
Ning Xie committed over 1 year ago
[CI/Build] Fix torch nightly CI dependencies (#19505)
Richard Zou committed over 1 year ago
[UX] Add Feedback During CUDAGraph Capture (#19501)
Robert Shaw committed over 1 year ago
rasmith committed over 1 year ago
[Kernels] Add activation chunking logic to FusedMoEModularKernel (#19168)
bnellnm committed over 1 year ago
[Docs] Remove WIP features in V1 guide (#19498)
Woosuk Kwon committed over 1 year ago
[Misc] Fix misleading ROCm warning (#19486)
Jee Jee Li committed over 1 year ago
[doc] fix "Other AI accelerators" getting started page (#19457)
David Xia committed over 1 year ago
[CI] Update FlashInfer to 0.2.6.post1 (#19297)
Michael Goin committed over 1 year ago
[Bugfix] Update the example code, make it work with the latest lmcache (#19453)
runzhen committed over 1 year ago
Support no privileged mode on CPU for docker and kubernetes deployments (#19241)
Louie Tsai committed over 1 year ago