A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 6, 17:00 UTC
main
[Model Runner V2] Add Support for XD-RoPE (#36817)
Santino Ramos committed 8 months ago
[Bugfix][Frontend] Fix audio transcription for MP4, M4A, and WebM formats (#35109)
seanmamasde committed 8 months ago
[Misc] Clean up Kimi-audio whisper encoder loading (#36903)
Isotr0py committed 8 months ago
Julien Denize committed 8 months ago
Sergey Zinchenko committed 8 months ago
[CI] Shard Multi-Modal Models (Standard) into 4 parallel jobs (#37014)
Kevin H. Luu committed 8 months ago
Enable loading of fused expert weights in the Transformers modelling backend (#36997)
Harry Mellor committed 8 months ago
[CI] Split Distributed Tests (4 GPUs) into 3 parallel jobs (#37015)
Kevin H. Luu committed 8 months ago
[Refactor] Relocate chat completion and anthropic tests (#36919)
Flora Feng committed 8 months ago
[CI] Pin helion version (#37012)
Yanan Cao committed 8 months ago
[V1] Remove pin_memory() in async_copy_to_gpu to fix sporadic stalls (#37006)
sbeurnier committed 8 months ago
[UX] Improve UX of CPU backend (#36968)
Li, Jiang committed 8 months ago
[responsesAPI] prioritize content over summary in reasoning item input (#36516)
Andrew Xia committed 8 months ago
fix: resolve chat template names before kwargs detection (#36937)
Giulio Leone committed 8 months ago
[Feat][Bugfix] Enable additional dimension for Flashinfer MLA and fix routing dtype (#36931)
Dimitrios Bariamis committed 8 months ago
[Bugfix] Fix MLA attention crash with AWQ/GPTQ quantized models (#34695)
haosdent committed 8 months ago
[Refactor] Consolidate SupportsEagle (#36063)
Benjamin Chislett committed 8 months ago
[BugFix] Fix "DP Coordinator receives unexpected..." messages (#37008)
Nick Hill committed 8 months ago
[Bugfix] Fix DeepSeek-V3.2 tokenizer stripping spaces (#37004)
Matthew Bonanni committed 8 months ago
[CI] Split V1 e2e + engine (1 GPU) into separate jobs (#36945)
Kevin H. Luu committed 8 months ago