A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Refactor] Remove multiple dead codes (#50285)
Wentao Ye committed 3 months ago
[Bugfix] Remove bad startup assertion (#50869)
Benjamin Chislett committed 3 months ago
[CI] Mooncake PD integration tests (#46844)
Nicolò Lucchesi committed 3 months ago
docs: document `reasoning_content` output removal as a breaking client change (#50624)
Fede Kamelhar committed 3 months ago
[Model] Add K-EXAONE-2.0-750B-A37B (#50524)
Kyungmin Lee committed 3 months ago
Support quantized DSpark Markov heads (#50424)
Andrii Skliar committed 3 months ago
[CI] KimiLinear PD in nightlies (#50266)
Nicolò Lucchesi committed 3 months ago
[CPU] Fix torch.compile crash from torch.accelerator.synchronize on CPU-only hosts (#49960)
Ganesh R committed 3 months ago
[UX] remove torch compile warning when using breakable cudagraph (#50750)
Jiangyun Zhu committed 3 months ago
[CI] And PPL test for multimodal generation models (#50839)
wang.yuqi committed 3 months ago
[CPU] Refine CPU kernel dispatch (#50801)
Li, Jiang committed 3 months ago
K3: Move LatentMoERunner (#50678)
Jee Jee Li committed 3 months ago
fix: remove stray duplicate from serving benchmark config (#46870)
Christopher Miyai committed 3 months ago
[Perf] Speed up multimodal placeholder and token-match scanning (#50716)
Hareesh Gali committed 3 months ago
fix(security): classify DeepStream as GPU backend and enforce pixel limits (#50755)
Juan Pérez de Algaba committed 3 months ago
Shard the K3 Latent-MoE up-projection on large batches (#50383)
Jee Jee Li committed 3 months ago
[CPU] Migrate unquantized MoE to the modular-kernel experts structure (#50133)
Li, Jiang committed 3 months ago
[Frontend] Require cache_salt to be non-empty via schema (#50816)
Cyrus Leung committed 3 months ago
[XPU] [Linear] add torch as xpu linear backend (#49664)
zofia committed 3 months ago
Qiming Zhang committed 3 months ago