A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 8, 08:45 UTC
main
[Docs] Spec decoding docs warning removal (#34439)
Nicolò Lucchesi committed 9 months ago
[BUG] Reset running requests when clearing cache for pause/resume (#34382)
Aaron Hao committed 9 months ago
Add config file for fused MoE for Nemotron (TP4, B200) (#34411)
danisereb committed 9 months ago
[ROCm] Enable MXFP4 MoE weight pre-shuffling on gfx950 and update aiter (#34192)
Douglas Lehr committed 9 months ago
[V0 Deprecation] Remove code related to per-request logits processors (#34400)
Cyrus Leung committed 9 months ago
Fix Mistral config remap to accept compressed-tensors quantization #34028 (#34104)
baonudesifeizhai committed 9 months ago
Vllm CPU benchmark suite improvement (#34128)
Louie Tsai committed 9 months ago
[bugfix] refactor FunASR's _get_data_parser (#34397)
AllenDou committed 9 months ago
[Bugfix] Fix Sparse24 Compressed Tensors models (#33446)
Kyle Sayers committed 9 months ago
Fix DeepSeek-OCR tensor validation for all size variants (#34085)
Yichuan Wang committed 9 months ago
[BugFix] Fix DP chunking (#34379)
Lucas Wilkinson committed 9 months ago
[Refactor] Pass Renderer to Input Processor (#34329)
Cyrus Leung committed 9 months ago
[Refactor] Move validation to params definitions (#34362)
Cyrus Leung committed 9 months ago
[Bug Fix] Fix `naive_block_assignment` always defaulting to False due to arg misalignment (#33848)
Runkai Tao committed 9 months ago
[Bugfix] Fix MTP accuracy for GLM-5 (#34385)
Michael Goin committed 9 months ago
[Refactor] Replace `activation: str` with `MoEActivation` enum (#33843)
Michael Goin committed 9 months ago
[ci] Integrate AMD tests into CI (#33626)
Kevin H. Luu committed 9 months ago
[ROCm][CI] Revert Test Groups From mi325_8 to mi325_1 Agent Pool In AMD CI (#34384)
Micah Williamson committed 9 months ago
[Bugfix] Fix some issues with MoERunner PR #32344 (#34371)
bnellnm committed 9 months ago
Fix CI failure - Flashinfer Kernel tests (#34316)
Wei Zhao committed 9 months ago