A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 8, 15:15 UTC
main
[Bugfix] Voxtral prompt/audio placeholder alignment (#34140)
Artus Krohn-Grimberghe committed 9 months ago
[Bugfix] Avoid duplicate k-proj weight emission in helper (#34142)
Artus Krohn-Grimberghe committed 9 months ago
[Kernel] use flashinfer for gdn prefill (#32846)
Jiangyun Zhu committed 9 months ago
[Bugfix] Fix shared expert input for latent MoE in EP+DP (Nemotron-H) (#34087)
TomerBN-Nvidia committed 9 months ago
[Kernel] FlashInfer: switch allreduce fusion to unified API (#33985)
Mohammad Miadh Angkad committed 9 months ago
Add NUMA Core binding in nixl_connector for CPU xPyD (#32365)
ZhengHongming888 committed 9 months ago
[CI][torch.compile] Fix incorrect filtering for E2E fusion tests on B200 (#34031)
Luka Govedič committed 9 months ago
[UX] Add `--language-model-only` for hybrid models (#34120)
Roger Wang committed 9 months ago
[Misc] Fix up attention benchmarks (#33810)
Lucas Wilkinson committed 9 months ago
[MODEL] Adding Support for Qwen3.5 Models (#34110)
JJJYmmm committed 9 months ago
[XPU][6/N] add xpu scaled_mm kernel (#34117)
zofia committed 9 months ago
[Fix] [CPU Backend] : Prepack weights for w8a8 oneDNN matmul (#33901)
Nikhil Gupta committed 9 months ago
[ASR] Fix audio benchmark and add RTFx metric (#32300)
Ekagra Ranjan committed 9 months ago
[CI] Remove empty image_size_factors for fuyu, glm4_1v, glm_ocr (#34107)
Andreas Karatzas committed 9 months ago
[Model] GLM adaptation (#34124)
Jee Jee Li committed 9 months ago
fix(cpu): fix mla_decode compilation on x86 without AVX512 (#34052)
ihb2032 committed 9 months ago
[BugFix] Fix `fastsafetensors` TP all procs using all GPUs (#34070)
Nick Hill committed 9 months ago
[Frontend][last/5] Make pooling entrypoints request schema consensus. (#31127)
wang.yuqi committed 9 months ago
[Tiny] Rename encoder budget file to more specific name (#34103)
Reagan Lee committed 9 months ago
[bug-fix] supported_tasks is breaking backward compatibility at init_app_state (#34027)
kourosh hakhamaneshi committed 9 months ago