A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Doc] Switch K8S examples to default MP mode (#39400)
Peter Pan committed 5 months ago
Wentao Ye committed 5 months ago
[Bugfix] [DSV4] [ROCm] Pin apache-tvm-ffi version to `0.1.10` (#45169)
TJian committed 5 months ago
[Bugfix] Fix Llama4 weight loading (#45047)
Tyler Michael Smith committed 5 months ago
[Bugfix] Fix tool parsing crash with non-function tool types (e.g. WebSearchTool) (#45147)
Ben Browning committed 5 months ago
[Bugfix] Fix nemotron accuracy drop introduced by #41184 (#45037)
bnellnm committed 5 months ago
[Hybrid] Marconi-style admission policy for hybrid cache (#37898)
Stan Wozniak committed 5 months ago
[Test] Fix one-sided MNNVL alltoall test workspace under-reservation (#44946)
Yongye Zhu committed 5 months ago
[CI] Ping Mistral team for ministral/voxtral/mixtral/pixtral changes (#45153)
Julien Denize committed 5 months ago
[MRV2][Spec Decode] DFlash (#44586)
Benjamin Chislett committed 5 months ago
fix: prefix DeepSeek V4 MTP projections (#44821)
Yufeng He committed 5 months ago
fix: AOT compile cache collision for dataclass-based HF configs (#45059)
Angela Yi committed 5 months ago
[bugfix] skip conch kernel for g_idx reordering (#45072)
Divakar Verma committed 5 months ago
KV-Cache multi-tier offloading async batched lookup (#44193)
Effi Ofer committed 5 months ago
Deprecated 1st generation Qwen and QwenVL models (#45131)
Harry Mellor committed 5 months ago
[Bugfix] Fix missing sequence_lengths in EXAONE-4.5 vision encoder (#45073)
Jongsu Liam Kim committed 5 months ago
[Frontend][Metrics] Add `vllm:tool_call_parser_invocations_total` Prometheus metric (#44448)
yzong-rh committed 5 months ago
[Bugfix] Fix layerwise reload dropping params after a composed weight loader (#44814)
hallerite committed 5 months ago
[Perf] Fix dsv3_router_gemm heuristic (#44217)
Roberto L. Castro committed 5 months ago
[Bugfix] Lazily import the humming quantization backend (#44921)
Michael Goin committed 5 months ago