A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Doc] Fix dead link in KV transfer README (#53230)
Harish Gaikwad committed 2 months ago
[Bugfix] Fix int32 index overflow in LoRA punica kernels at long context (#53034)
Shuai Shao committed 2 months ago
[XPU] follow cuda path for mrope on XPU (#53201)
Yan Ma committed 2 months ago
[Bugfix][LoRA] Use an explicit capability flag for tower connector LoRA (#53092)
linitra24 committed 2 months ago
[XPU][CI]Add more cases in intel GPU CI and reorganize to align non-xpu part (#51630)
xiangdong committed 2 months ago
[XPU][CI]Add parallelism for long-running Intel GPU cases (#52257)
xiangdong committed 2 months ago
[XPU][INC] Add int4 w4a8 (dynamic int8 activation) backend for INC linear layers (#50501)
Thanaji Rao Thakkalapelli committed 2 months ago
Support kimi k3 nvfp4 checkpoint (#53132)
Wei Zhao committed 2 months ago
[Spec Decode] DFlash2: local convolution + candidate selector (#52816)
Zihan Zhang committed 2 months ago
Roy Wang committed 2 months ago
[Bugfix] Fix batch-invariant fp32 matmul OOR on SM89 for N=1 (#52960)
vhagor committed 2 months ago
[Kimi-K3] Extend GEMM-RS to GEMM-AR (#53053)
Thien Tran committed 2 months ago
[MM] Remove text components from ProcessorInputs (#53093)
Cyrus Leung committed 2 months ago
[Rust Frontend] Fix Kimi K3 reasoning_effort="none" handling (#53043)
Reid committed 2 months ago
[Spec Decode] Enable adaptive verification on DSv4 + sm90 (#52795)
Lucas Wilkinson committed 2 months ago
[Rust Frontend] Add HY3 unified parser and local XGrammar structural-tag builder (#53054)
Bugen Zhao committed 2 months ago
[Bugfix][Structured Output] Avoid spurious FSM errors after speculative reasoning end (#53046)
Chauncey committed 2 months ago
[Rust Frontend] Replace external `protoc` with pure Rust lib `protox` (#52892)
Bugen Zhao committed 2 months ago
[Bugfix][Spec Decode] Scope DSpark backend inheritance to DeepSeek V4 (#52809)
Misha Goin committed 2 months ago
[Bugfix] Load untied Gemma LM head weights (#53170)
Kevin H. Luu committed 2 months ago