A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Bug] Fix tmp directory for `lm_eval` (#47755)
Wentao Ye committed 4 months ago
Ameen Patel committed 4 months ago
AGENTS MD: Add suggestion on how to incorporate tests (#47784)
Simon Mo committed 4 months ago
[XPU] Fix Event init failure w/ blocking (#47868)
liuzhenwei committed 4 months ago
Enable causal masking for SWA in vllm-project/speculators models (#47745)
Eldar Kurtić committed 4 months ago
[kv_offload] Establish tier-owned KV event handling (#46544)
Chang Guo committed 4 months ago
[Bugfix] Fix PD disagg + MTP correctness for Qwen3.5(GDN) (#47466)
Dakai An committed 4 months ago
[KV-Offloading] Support workload identity for objectstore secondary tier (#47063)
Pierangelo Di Pilato committed 4 months ago
Disable dynamic speculative decoding when DP is enabled (#45963)
Tyler Michael Smith committed 4 months ago
[Bugfix] Fix UBatchWrapper CUDA graph key to sum all ubatches, not just first two (#43161)
Lanze Liu committed 4 months ago
[UX] Log worker exit code when process dies unexpectedly (#38641)
Nick Cao committed 4 months ago
[Bugfix][Core] Close underlying iterator in merge_async_iterators single-iterator fast path (#44726)
Ting SUN committed 4 months ago
Bump Transformers version to 5.10.4 (#41359)
Harry Mellor committed 4 months ago
fix: ensure no double load of lm head in nemotron mtp (#47440)
Shaun Kotek committed 4 months ago
[BugFix] Fix ModelOpt mixed-precision quantization for sparse `quantized_layers` configs. (#47318)
danielafrimi committed 4 months ago
[CI] Fix Transformers modeling backend LoRA test (#47832)
Harry Mellor committed 4 months ago
[Bugfix] Reject sampling params unsupported by diffusion models (#45418)
Guan-Ming Chiu committed 4 months ago
[KV Offload] Add `ParentManager` ABC for secondary tier callbacks (#47274)
Ronen Schaffer committed 4 months ago
[Bugfix][KV offload] Store interior chunk-boundary blocks under MTP/Eagle (#46972)
drakosha committed 4 months ago
[ROCm][CI] Refresh ROCm base images when docker rocm_base changes (#46904)
Andreas Karatzas committed 4 months ago