A high-throughput and memory-efficient inference and serving engine for LLMs
main
Validate scale-out multimodal data before engine handoff (#51898)
Clinton Thomas committed about 2 months ago
[Kernel] Add fused MoE tuned config for E=256,N=512 on NVIDIA A100 80GB PCIe (#55511)
Baki Burak Öğün committed about 2 months ago
[CPU] [Feat] Add native AMX-FP8 attention impl for Diamond Rapids (#49410)
zhao, zhenhui committed about 2 months ago
[Core] Sync DP state on the first step of a wave (#52957)
aoshen02 committed about 2 months ago
[Bugfix][Multimodal] Bound renderer warmup to the prefill token budget (#55448)
lucamotz committed about 2 months ago
[Kernel][HY V4] Add Triton iHC pre/post fallback (#55059)
linitra24 committed about 2 months ago
[Bugfix][Kernel] Build fused GDN MTP decode for SM110 (#53835)
WikAlie committed about 2 months ago
[Bugfix][Qwen4Exp] fix state index strides in fused PLE conv (#55375)
Harry Huang committed about 2 months ago
[Kernel] Fall back from persistent top-k on low-shared-memory GPUs (#54110)
lucamotz committed about 2 months ago
[Bugfix][DSv4] Seed the -1 sentinel in the prefill sparse index workspace (#55299)
aoshen02 committed about 2 months ago
[Bugfix] complete VLLMValidationError migration in chat_utils.py (#50254)
Adababy committed about 2 months ago
[Docs][Models] Use the official FunASR Nano vLLM checkpoint (#54944)
zhifu gao committed about 2 months ago
[Perf][Multimodal] Avoid duplicate text embedding in Qwen2.5-Omni (#55415)
waizuichougou committed about 2 months ago
[Feat] Add EPLB support for GLM-5.3-Flash (#55119)
Chauncey committed about 2 months ago
[Security] Validate cache salts before they reach LMCache (#51444)
Clinton Thomas committed about 2 months ago
[Attention] Sync FA with upstream (#54819)
GroupieSteven committed about 2 months ago
[Core][MRV2] Support eagle3 spec decode with pipeline parallel (#50514)
yongqinwang-cmd committed about 2 months ago
[CI][ROCm] Disable Transformers nightly groups (#55409)
Andreas Karatzas committed about 2 months ago
[CI][ROCm] Restore Wikitext coverage for Qwen OCP-MX (#55410)
Andreas Karatzas committed about 2 months ago
[CI][ROCm] Raise AMD job and server readiness timeouts (#55308)
Andreas Karatzas committed about 2 months ago