[Bugfix][KV Offload] Centralize shared mmap cleanup in CPU worker (#51622)

AlexHuang committed 3 months ago

[Bugfix][Core] Preserve Mamba running CoW after external hits (#51766)

Dao007forever committed 3 months ago

profiler: add PrivateUse1 activity support for custom backends (#50977)

David Holtz committed 3 months ago

[Perf] Avoid repeated multimodal prompt update scans (#51774)

Tianyu Guo committed 3 months ago

[XPU] [Linear] enable torch linear backend for blockwise gemm on xpu (#50826)

zofia committed 3 months ago

[Doc] Fix typos in speculative decoding docs (#51500)

Kyungmin Lee committed 3 months ago

[MRV2][Spec] Fuse AR speculator multi-step decodes back into one CUDA graph (#46849)

Yizhou committed 3 months ago

[ROCm][DSV4] Preserve native MXFP4 TP8 shard allocation (#51473)

Fangzhou Ai committed 3 months ago

Fix docs on `main` (#51773)

Harry Mellor committed 3 months ago

[CPU][Zen] Route BF16 MoE inference through zentorch on AMD (#44201)

Priyansh Jain committed 3 months ago

[XPU] install xpu-manager for device monitor (#50831)

Yan Ma committed 3 months ago

[CI][AMD] Persist the openai-harmony tiktoken vocab cache across jobs (#51666)

stefankoncarevic committed 3 months ago

[Attention] Fix MLA prefill workspace allocation size (#51733)

Wei Zhao committed 3 months ago

[Kernel] Optimize long-context MLA cache gathers (#51739)

Yongye Zhu committed 3 months ago

[Bugfix][Frontend] Report Cohere stop sequences correctly (#51556)

cherry77-cloud committed 3 months ago

replace batch_norm to numerically identical without cudnn (#51734)

Khushali Desai committed 3 months ago

Preserve revision pins in secondary artifact loaders (#51446)

Clinton Thomas committed 3 months ago

[Misc] Use VLLMValidationError in scoring input validation (#51753)

Frank committed 3 months ago

[BugFix][Core] free_blocks: restore prepend (LIFO) reuse order when prefix caching is off (#51482)

Ming Huang committed 3 months ago

[Perf] Adaptive budget for spec scheduled input tokens, ~60% better Kimi K3 DSpark TTFT (#51725)

Wentao Ye committed 3 months ago