A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Bugfix] Skip VllmConfig re-validation for with_hf_config submodel views (#58212)
Nick Hill committed 29 days ago
[ROCm] Use silu_and_mul_with_clamp's torch._C op (#52052)
Tres committed 30 days ago
Jack Hu committed 30 days ago
[Bugfix][ROCm] Use the platform FP8 range in the concat MLA q test (#58153)
stefankoncarevic committed 30 days ago
rasmith committed 30 days ago
[Perf] Remove CPU-GPU sync in heterogeneous vocabulary speculative decoding (#57396)
Michael Lapshin committed 30 days ago
[CI] Shard (H200 MIG 18GB) Spec Decode Draft Model across whole-directory replicas (#58193)
Thang Nguyen committed 30 days ago
[CI] Split LM Eval TurboQuant KV Cache into per-config jobs (#57113)
Thang Nguyen committed 30 days ago
[Core] Disable JIT warmup in eager mode (#58197)
Misha Goin committed 30 days ago
[Bugfix] Backport Inductor custom-op pattern matching fix (#58189)
Misha Goin committed 30 days ago
[CI] Build the torch-nightly image on Ubuntu 24.04 (#58204)
Andrey Talman committed 30 days ago
[Kernel] Remove AllSpark INT8 W8A16 GEMM backend (#58001)
Misha Goin committed 30 days ago
[Core] structured generation mode for DiffusionGemma model (Jev-like) (#57250)
Matt Mastracci committed 30 days ago
[Refactor] Remove dead code multiple places (#58002)
Wentao Ye committed 30 days ago
[Bugfix][Model][Spec Decode] Defer disposable GLM MTP head (#55442)
lucamotz committed 30 days ago
[Bugfix][V1] Read ModelState max_model_len from model config (#58149)
hcl committed 30 days ago
[ROCm][CI] Add GELU activation for AiterExperts in the modular-kernel coverage (#58030)
Divakar Verma committed 30 days ago
[Bugfix][V1] Honor enable_jit_warmup for V2 kernel warmup (#55146)
pengyihang committed 30 days ago
[MRV2] Release weight offloader on shutdown (#57834)
Taneem Ibrahim committed 30 days ago
[Bugfix] batch_invariant: keep non-AllReduce collectives enabled on NCCL >= 2.31 (#58179)
Guanxin Li committed 30 days ago