A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Model] DeepSeek-V4.1-Flash Model Definitions (#56228)
Roger Wang committed about 1 month ago
[Doc] Sync KV event medium terminology after #48123 (#51646)
AlexHuang committed about 1 month ago
[NIXL][PCP] Report replicated-PCP ranks > 0 as done sending instead of hiding them (#53903)
Lucas Wilkinson committed about 1 month ago
[Docs][Security] Clarify reporter credit and CVE publication timing (#55476)
Juan Pérez de Algaba committed about 1 month ago
[Kimi K3 Perf] Avoid KDA mixed-batch gather/scatter, 5.2%~7.7% E2E Throughput Improvement (#56159)
Wentao Ye committed about 1 month ago
[XPU][Bugfix] Add forward_xpu to Ernie4_5_VLRotaryEmbedding (#55942)
Jakub Byczkowski committed about 1 month ago
[XPU][Bugfix] Fix FalconH1 pipeline-parallel execution (#55913)
Jakub Byczkowski committed about 1 month ago
[Model] Add Bailing V3 VL support (#55921)
zexplorerhj committed about 1 month ago
[ROCm] Add better kv dtype error discoverability (#55236)
Giuseppe Grossi committed about 1 month ago
Toby Mao committed about 1 month ago
Jared Wen committed about 1 month ago
[Model][Frontend] Support DeepSeek-V4.1-Flash in Rust and Python frontends (#56208)
Bugen Zhao committed about 1 month ago
[ROCm][Perf] Kimi-K3 Fused kernels for KDA prefill reland (#54038)
kliuae committed about 1 month ago
[Kernel] Optional Q-norm in fused DSv4 MLA epilogue; group_size=32 for packed FP8 quant (#56215)
Yongye Zhu committed about 1 month ago
[Bugfix] Tolerate misspelled DSML tool_calls wrapper (#56141)
Flora Feng committed about 1 month ago
[Bugfix][MRV2] Initialize DCP metadata after batch partitioning (#55212)
Sy03 committed about 1 month ago
[multimodal][feat] add torchaudio backend to AudioResampler (#52598)
Jared Wen committed about 1 month ago
[ROCm][Perf] Route large DSV4 sparse prefill to AITER OPUS (#54855)
jiacao-amd committed about 1 month ago
[MM][CG] Enable encoder CUDA Graph for MiniCPM-V (#42785)
Yunzhu Lu committed about 1 month ago
[Model] Add DeepSeek-V4 CPU backend (#55355)
Li, Jiang committed about 1 month ago