A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 5, 19:45 UTC
main
Nicolò Lucchesi committed 7 months ago
[ROCm] [DOC] Update the Documentation to include ROCm Nightly Wheel support (#38457)
TJian committed 7 months ago
[CI] Fix SPLADE pooler test broken by #38139 (#38495)
haosdent committed 7 months ago
[ROCm] Enable MORI EP for unquantized MoE with AITER backend (#37529)
Tan Pin Siang committed 7 months ago
(security) Fix SSRF in batch runner download_bytes_from_url (#38482)
Juan Pérez de Algaba committed 7 months ago
[Bugfix][Model] Fix PixtralForConditionalGeneration LoRA (#36963)
Jee Jee Li committed 7 months ago
[Model][Quantization] Add GGUF support for MiniMax-M2.1 (#36965)
PikaPikachu committed 7 months ago
[ci] Soft fail and disable retry for AMD build image job (#38505)
Kevin H. Luu committed 7 months ago
[MoE] Add RoutingMethodType.Simulated to TRT-LLM FP8/NVFP4 kernel allowlists (#38329)
Jaewon committed 7 months ago
[Misc] Always use `forward_mulmat` for `Conv3d` on newer versions of torch. (#38487)
Roger Wang committed 7 months ago
[CI] Add temperature=0.0, reduce max_tokens, and add debug prints to audio_in_video tests (#38492)
Andreas Karatzas committed 7 months ago
Add @ZJY0516 to CODEOWNERS (#38497)
Jiangyun Zhu committed 7 months ago
[Bugfix] Support multi-type params parsing for DeepSeek v3.2 (#33703)
Stanislav Kirillov committed 7 months ago
[ROCm][CI] Enable hybrid chunked prefill test (#38317)
Andreas Karatzas committed 7 months ago
[QeRL] Fix online quantized reloading (#38442)
Kyle Sayers committed 7 months ago
Wentao Ye committed 7 months ago
[Transformers v5] fix missing pixtral/voxtral multimodal dispatch (#38410)
allgather committed 7 months ago
[ROCm][CI] Fix cross-attention dispatch for encoder-decoder models (#38450)
Andreas Karatzas committed 7 months ago
[Spec Decode, BugFix] Propagate norm_before_fc from Eagle3 speculator (#38111)
Shubhra Pandit committed 7 months ago
[Feature] Add Qwen3-ForcedAligner support via token classification pooling (#35367)
haosdent committed 7 months ago