A high-throughput and memory-efficient inference and serving engine for LLMs
main
[DeepEPv2] Support MXFp8 Activation Scale Dispatch (#51398)
Robert Shaw committed about 2 months ago
[Doc] Fix local input path in run-batch examples across docs (#53220)
QWERQWERQWE86 committed about 2 months ago
[MoE] enable all2all fi_one_sided by default (#53311)
Artem Perevedentsev committed about 2 months ago
[CI/Build] Improve pre-commit fail message (#53866)
Cyrus Leung committed about 2 months ago
[Offloader] Offload submodules that make_layers never reaches (#53120)
ray24777 committed about 2 months ago
[CI] forward fix CRCR report step in the torch-nightly lane (#53732)
Andrey Talman committed about 2 months ago
[CI] Build mamba-ssm with C++20 for torch 2.14 nightly compatibility (#49600)
Andrey Talman committed about 2 months ago
[Bugfix][Processor] Replace bare asserts with ValueError in DeepseekVLV2/OCR processors (#53854)
adisivaprasad committed about 2 months ago
[AttentionBackend][HPC-ops] update hpc rope norm to support stride kv cache (#53705)
Cheng Jiang committed about 2 months ago
[LoRA] Cleanup VocabParallelEmbedding (#53843)
Jee Jee Li committed about 2 months ago
[Config] Delegate PCP compatibility checks to PCP manager (#53853)
Qiu Chunshuo committed about 2 months ago
[Bugfix][Mooncake] Fix Mamba prefill truncation ordering (#53663)
Ziming Huang committed about 2 months ago
rasmith committed about 2 months ago
[Hardware][AMD][Perf][Bugfix] Update ROCr and clr in base image (#53712)
Matt committed about 2 months ago
[RL] Add rank-local IPC weight updates (#52497)
aoshen02 committed about 2 months ago
[Bugfix][Model] Honor Molmo2 dummy video num_frames >= 2 override (#53830)
Hung Hoang (黄兴) committed about 2 months ago
BugFix(PCP): use persistent input buffers for PIECEWISE CUDA graphs (#53515)
Qiu Chunshuo committed about 2 months ago
[Bugfix][Structured Output] Preserve reasoning adapters for shared parser engines (#52830)
Flora Feng committed about 2 months ago
[Kernel][Perf] Tune fused_moe FP8 config for Qwen3.5 on L40S (+7%) (#53819)
nicholaskh-ai committed about 2 months ago
[Frontend] Add `/v1/messages/render` endpoint for the Anthropic Messages API (#45803)
Hyunkyun Moon committed about 2 months ago