A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 4, 05:00 UTC
main
fix(openai): tolerate empty content in forced tool choice (#40148)
jack committed 6 months ago
nixl refactor: new transfer design (#40731)
zhanqiuhu committed 6 months ago
[MM][Gemma4] Respect max_soft_tokens in encoder budget (#41799)
SeongJun Lee committed 6 months ago
[CPU][RISC-V] Auto-bind OMP threads and harden nobind path (#40569)
lyd1992 committed 6 months ago
Fix some legacy checkpoints with deprecated `rope_type` values (#41734)
Harry Mellor committed 6 months ago
[Bugfix][Rocm]Aiter MoE re-uses existing tensor addresses after weight update. (#40390)
Yuankai Chen committed 6 months ago
[Bugfix][CI] Fix Disaggregated test area path (#41794)
Nicolò Lucchesi committed 6 months ago
fix(rocm): remove workaround causing invalid argument on Qwen3.5 with TP=2 (#40686)
Zhaodong Bing committed 6 months ago
[XPU] Disable CUDA graph memory estimate on XPU platform (#41344)
Chaojun Zhang committed 6 months ago
[Examples] Resettle Disaggregated examples. (#40759)
wang.yuqi committed 6 months ago
[Bugfix] Fix codegen for unqualified names (#40726)
Lucas Kabela committed 6 months ago
[Feat] dnnl build for AVX2 W8A8 Int8 (#41318)
Tianmu Li committed 6 months ago
[CPU] Add FP8 W8A16 linear support (#41186)
Yuwen Zhou committed 6 months ago
[Bugfix] Skip PP sampled-token receive on last rank during async scheduling (#40749)
wi-adam committed 6 months ago
[Spec Decode] Allow multimodal models with a warning (#41752)
Li committed 6 months ago
[ROCm][CI] Refine gating tests (#37243)
Andreas Karatzas committed 6 months ago
[Fix] Add missing stubs from cpu fp8 attention changes (#41387)
Tianmu Li committed 6 months ago
[Build] Fall back to system libgomp when torch has no vendored copy (#40575)
lyd1992 committed 6 months ago
[Mistral Tokenizer] allow more leniency in apply_chat_template (#41658)
Julien Denize committed 6 months ago
[Frontend] Supports resubmitting output items with missing fields in Responses API (#41355)
Chauncey committed 6 months ago