A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Bugfix][NIXL] Avoid receive reports for notification-only requests (#57570)
Nicolò Lucchesi committed about 1 month ago
[Model] DiffusionGemma: honor logprob_token_ids on the converging step (#57417)
Matt Mastracci committed about 1 month ago
[Perf][DSV4.1] Optimize MegaMoE staging and NVFP4 cache gathers (#57604)
Woosuk Kwon committed about 1 month ago
[KV Offloading] Back-pressure detection and remediation (#50045)
bnellnm committed about 1 month ago
[ROCm][DSv4] Fix sparse-indexer logits collapse on gfx950/gfx942 (#50455)
frida-andersson committed about 1 month ago
[Multimodal] Type dummy options per modality (#57576)
Harry Mellor committed about 1 month ago
[Deprecation] Remove assistant_token_mask support (#57520)
Cyrus Leung committed about 1 month ago
[Nixl] Separate transport-failure metrics from KV expiry (#55854)
Nicolò Lucchesi committed about 1 month ago
[Core] Make parallel sampling (n>1) reqs admission atomic (#53936)
Nicolò Lucchesi committed about 1 month ago
[XPU] sleep mode: fix KV cache release test (#57485)
Yan Ma committed about 1 month ago
[DeepSeek V4] Fix fused MoE expert distribution (#57465)
Itay Alroy committed about 1 month ago
[Misc] Remove unnecessary Transformers version guards (#57556)
Cyrus Leung committed about 1 month ago
[Quantization][XPU] Enable int8_w8a8 MoE on the Triton backend for XPU (#53162)
Artur Fierka committed about 1 month ago
[Frontend] Add stream reasoning and tool calls from the derender endpoint (#50550)
Martin Hickey committed about 1 month ago
[Distributed][MoonEP] BF16 integration of MoonEP balanced EP backend (#52101)
Kevin Li committed about 1 month ago
[fix] Mistral-Large-3 accuracy regression on `main` (#57563)
Julien Debache committed about 1 month ago
[Quantization] Support kimi-k3 routed expert quant (#57430)
Kyle Sayers committed about 1 month ago
[Model][Gemma4] Load Weights with AutoWeightsLoader (#55911)
Asaf Gardin committed about 1 month ago
[Mypy] Fix mypy typing for N/O models (#54142)
Taneem Ibrahim committed about 1 month ago
[Tokenizer] Drop dead Mistral tokenizer shims for transformers#41962 (#56325)
Aditya Ganesh Kumar committed about 1 month ago