A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 4, 12:30 UTC
main
build: embed image provenance metadata in vLLM containers (#40653)
Alec committed 6 months ago
[Bugfix] DSV32/V4 add missing type conversion for non-streaming tool calls (#41198)
Chauncey committed 6 months ago
Chauncey committed 6 months ago
[CI][CPU] Split CPU-Distributed Tests into per-scenario labels (#41203)
haosdent committed 6 months ago
[CI/Build] Auto-detect manylinux ABI tag for nightly wheels (#41149)
Shengqi Chen committed 6 months ago
[CI] fix test_rotary_embedding_opcheck format error (#41202)
Chauncey committed 6 months ago
[Reasoning] Support for speculative decoding with thinking budget (#34668)
rishitdholakia13 committed 6 months ago
[KV Offload] Per-job store completion for CPU offloading connector (#39186)
Itay Etelis committed 6 months ago
Jee Jee Li committed 6 months ago
[Bugfix] Fix max_num_batched_token not captured in cuda graph (#40734)
Wei Zhao committed 6 months ago
[Bugfix] fix inductor error for dpsk v4 (#41135)
Jiangyun Zhu committed 6 months ago
[BugFix][CPU] fix error on CPU runner shutdown (#41034)
Fadi Arafeh committed 6 months ago
[Bugfix] Fix Deepseek V4 import error due to AOT compile cache loading (#41090)
Wei Zhao committed 6 months ago
[Feature] add cohere reasoning and tool parsers (#40422)
Walter Beller-Morales committed 6 months ago
[BE][Torch 2.12] Remove workaround code for fixed cublas issue (#40845)
Lucas Kabela committed 6 months ago
[QeRL] Add warnings for extra memory buffering (#40309)
Kyle Sayers committed 6 months ago
[Docs] [QeRL] Layerwise Reloading Documentation (#40317)
Kyle Sayers committed 6 months ago
uncomment flex backend for batch invariant mode (#40842)
liangel-02 committed 6 months ago
[UX] Allow enable/disable model weights loading tracking by config (#41086)
Isotr0py committed 6 months ago
[CI] De-flake test_chat_completion_n_parameter_non_streaming (#41147)
haosdent committed 6 months ago