A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Bugfix][Frontend] Force reasoning mode for GLM-5.3 chat templates in the GLM MoE parser (#56994)
Shijin Zhang committed 22 days ago
[Frontend] Support strict MiMo-V2.6 tool calling (#58019)
Yixin Dong committed 22 days ago
[Config] Remove Engram CUDA-alike device restrictions (#59171)
jack committed 22 days ago
[Docs] Speed up docs build ~5x (#58705)
Harry Mellor committed 22 days ago
[ROCm][CI] Mirror generic GEMM-RS/AR on MI355 (#58433)
Andreas Karatzas committed 22 days ago
[Bugfix][Rust Frontend] Account for new requests in DP routing (#58956)
Reid committed 22 days ago
[Bugfix][Rust Frontend] Fix startup with config-only model (#58643)
Louie Tsai committed 22 days ago
[Fast Start] Support PP (#55477)
siyu committed 23 days ago
[Bugfix][Model] MiMo: keep fused fp8 qkv_proj pairing state across weight-loading calls (#58142)
yhcheong (vllmellm) committed 23 days ago
[Bugfix][Core] Allow AuxOutput reset after abort without running requests (#59060)
aoshen02 committed 23 days ago
[Feat][Model] Enable KDA prefill checkpoints for GLM-5.3-Flash (#56960)
Chauncey committed 23 days ago
[Bugfix][ROCm] Keep Mooncake bootstrap ports bound during startup (#58967)
Andreas Karatzas committed 23 days ago
[ROCm][CI] Expand MI355 mirrors and route MIG-sized jobs to DPX (#59137)
Andreas Karatzas committed 23 days ago
[MM] Enable device normalization for Llama Nemotron VL Embed/Rerank (#57928)
Jakub Zakrzewski committed 23 days ago
[Misc] Avoid repeated warnings for merged Anthropic system messages (#59172)
Chauncey committed 23 days ago
[Model] Decoder-side SWA bounded replay for DeepSeek-V4.1 (#58132)
Yifan Qiao committed 23 days ago
[Bugfix][Rust Frontend] Support raise_exception in chat templates (#59020)
Yashasvi Asthana committed 23 days ago
[ROCm][Model][Bugfix] Fix GLM-5.2 shared-expert fusion and MTP on the ROCm DSA path (#58904)
Jin Tao committed 23 days ago
[DSv4.1] Avoid runtime recompiles of _ring_slot_mapping_kernel (#59119)
Juntian Liu committed 23 days ago
[Bugfix] Hoist $defs/definitions in Cohere parser tool schema composition (#49602)
Václav Čadek committed 23 days ago