A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Doc] docs: fix note formatting for pooling models (#47701)
Sungjae Lee committed 4 months ago
[XPU] Route mm_prefix models to Triton attention backend (#47688)
liuzhenwei committed 4 months ago
[CI/Build] Fix pre-commit check (#47695)
Li, Jiang committed 4 months ago
[Bugfix] Recycle post-final-norm hidden in GLM MTP (single norm) (#47448)
xiaozhoupy committed 4 months ago
[XPU][CI]Add agent tags for Basic Models Tests (Initialization) in Intel GPU CI (#47675)
xiangdong committed 4 months ago
[Core][DP] Rotate load-balancer tie-break to avoid systematic engine bias (#47420)
Qiming Zhang committed 4 months ago
[ROCm][CI] Increasing parallelism in Basic Models Tests (Extra Initialization) (#47591)
Andreas Karatzas committed 4 months ago
[ROCm][CI] Adding nixl multiconn (#47481)
Andreas Karatzas committed 4 months ago
[Frontend] Support OpenAI Responses API namespace tools (#47024)
Zhong committed 4 months ago
[RISC-V] Enable BF16 on VLEN=256 hardware (#45243)
velonica0 committed 4 months ago
[Performance][Hardware][RISC-V] Reduce LMUL pressure in INT4 LUT dequant (#47538)
LiuTong committed 4 months ago
[Bugfix][CPU][RISC-V] Fix VLEN detection for RVV attention path (#47532)
LiuTong committed 4 months ago
[INC][ARK] Direct Register Custom Op for ARK (#46361)
Zhenzhong Xu committed 4 months ago
[CI/Build][CPU] Remove global extra index (#47687)
Li, Jiang committed 4 months ago
[CPU][BugFix] Multiple fixes to w4a8_int8 CPU MoE path (#46739)
Fadi Arafeh committed 4 months ago
[XPU] limit max-num-seqs in test_lmeval.py for XPU (#47682)
Qiming Zhang committed 4 months ago
[Test][XPU] Skip fork in kv_sharing_fast_prefill test on XPU (#47406)
Liangliang Ma committed 4 months ago
[Test][LoRA] Use lightweight CPU reference and skip heavy cleanup in punica ops tests (#47534)
Chaojun Zhang committed 4 months ago
[Feature] Support MTP speculative decoding for Bailing hybrid models (#44880)
alex101-ops committed 4 months ago
[Attention Backend] HPC_ATTN backend support mtp and dynamic scheduled attention (#47433)
Cheng Jiang committed 4 months ago