A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Mypy] Fix mypy typing for Qwen and Qianfan models (#58046)
Ashraf Bhuiyan committed 26 days ago
[Bugfix][Frontend] Detect Anthropic inline-system merge against the resolved chat template (#58754)
Robert Shaw committed 26 days ago
[Bugfix] Fix Anthropic Thinking Disabled with P/D (#58786)
Robert Shaw committed 26 days ago
[Perf][MoE] Index expert mapping lookups in RoutedExperts.load_weights (#58720)
Willian Z committed 26 days ago
[Refactor] Remove dead tests utils (#58803)
Wentao Ye committed 26 days ago
[Elastic EP] Fix EPLB load statistics during scaling (#58473)
Itay Alroy committed 26 days ago
[Security] Gate per-request multimodal processor kwargs (#58830)
Juan Pérez de Algaba committed 26 days ago
[CI] Split (B200) Miscellaneous Kernels into mHC, FLA Ops and Misc named jobs (#58609)
Thang Nguyen committed 26 days ago
[Spec Decode] Enable Gemma4 DSpark adaptive verification with FlashInfer (#57263)
qizixi committed 26 days ago
[CI] Only isolate the registry tests that need a fresh process (#58764)
Aarushi Jain committed 26 days ago
[Bugfix][Frontend][Rust Frontend] Update DeepSeek V4.1 Flash reasoning effort mappings (#58316)
zhec committed 26 days ago
[Bugfix][ROCm] AMD-Quark mixed-precision DeepSeek-V4.1 support (#57071)
xiao-llm committed 26 days ago
[RL] Add sharding-aware NCCL M2N weight transfer (#51520)
Ke Wen committed 26 days ago
[Perf][DSv4.1] Shard the Engram wkv projection across TP ranks (#58678)
Shuolei Wang committed 26 days ago
[Core] Bound draft-token RPC waits by the execute-model timeout (#58779)
Chandra Mourya committed 26 days ago
[Bugfix][Frontend] Fix Inkling tool name leaking into content after reasoning (#58792)
Baljinder Hothi committed 26 days ago
[CI] Split (H200 MIG/MI300) Basic Correctness into named jobs (#57054)
Thang Nguyen committed 26 days ago
[Bugfix][Reasoning] Count Kimi K3 reasoning tokens (#58372)
Elvir Crnčević committed 26 days ago
[Bugfix] V1: fix allowed_token_ids_mask aliasing in InputBatch.swap_states (#48419)
Rui Zhu committed 26 days ago
fix: perf: use startswith(x, i) instead of string slicing to avoid O(N^2) (#52580)
Ricardo-M-L committed 26 days ago