A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Rust Frontend] Add standalone `granite4` tool parser (#45216)
Tahsin Tunan committed 5 months ago
[Docs][KV Connector][NIXL] document KV Transfer stat logging and Prometheus metrics (#44055)
Sai Sridhar Tarra committed 5 months ago
[Frontend] Support strict mode for tool calling with ResponsesAPI (#45396)
Chauncey committed 5 months ago
[Core][KV Connector] fix scheduler KV connector stats aggregation (#43877)
Srinivas Krovvidi committed 5 months ago
[Core] Support structured outputs for beam search (#35022)
Guan-Ming (Wesley) Chiu committed 5 months ago
[Bug] Migrate Reset cache for both v2 and v1 model runner (#42759)
Wentao Ye committed 5 months ago
[V1][Metrics] Add MLA attention metrics for DeepSeek MFU estimation (#39457)
Thillai Chithambaram committed 5 months ago
[Metrics] Add group-aware KV cache capacity to vllm:cache_config_info (#42206)
Ethan Feng committed 5 months ago
[BUGFIX][XPU] Update fa interface for compatibility (#45394)
liuzhenwei committed 5 months ago
[Model] Remove Mono-InternVL (InternLM2VEForCausalLM) (#45129)
Tiezhen WANG committed 5 months ago
[KV Connector]: Support KV push from Prefill to Decode node using Nixl KV Connector (#35264)
snadampal committed 5 months ago
[11b/n] Migrate Machete kernels to torch stable ABI (#45304)
Chris Leonard committed 5 months ago
[Doc] AGENTS.md: add section about coding style (#45301)
Thomas Parnell committed 5 months ago
Fix misleading error for audio duration limit rejection (#45113)
Juan Pérez de Algaba committed 5 months ago
Update hidden states extraction integration test triggers (#45294)
Fynn Schmitt-Ulms committed 5 months ago
[Bugfix][Rust Frontend] Return 400 for prompt-validation submit errors (#45286)
JinYan Su committed 5 months ago
[Frontend] Support strict mode for tool calling (#45003)
Chauncey committed 5 months ago
[CI] ci-fetch-log.sh: fetch all failed jobs from a build URL or PR number (#45274)
Michael Goin committed 5 months ago
[XPU][DeepSeek-V4] Fix MTP: sync with upstream fixes #44821 and #43746 (#45240)
Ma Jian committed 5 months ago
Will Eaton committed 5 months ago