A high-throughput and memory-efficient inference and serving engine for LLMs
main
[CI][ROCm] Avoid redundant image pulls during smoke validation (#54408)
Andreas Karatzas committed about 2 months ago
[Core][MRV2] Freeze gc during V2 CG capture; skip per-descriptor cleanup (#54646)
Nick Hill committed about 2 months ago
[Rust Frontend] Migrate to new tekken crate (#53056)
Jorge Menjivar committed about 2 months ago
[CI] Broaden tool-calling issue auto-labeling (#54650)
Flora Feng committed about 2 months ago
[Perf][Rust Frontend] Count the tokenizer vocabulary once at construction (#54449)
Salt Sato committed about 2 months ago
[Kernel] Make prefix-prefill tiling independent of the KV page size (#54194)
Zheng Gong committed about 2 months ago
[XPU] Route activation CustomOps to SYCL kernels (#53734)
Marceli Fylcek committed about 2 months ago
[CI] Build CPU image against torch nightly for TORCH_NIGHTLY runs (#48750)
Andrey Talman committed about 2 months ago
[XPU] bump up auto-round-lib to 0.15.0 (#54515)
Zhenzhong Xu committed about 2 months ago
[Bugfix][Rust Frontend] Fix adjacent DeepSeek V4 user content rendering (#53281)
Reid committed about 2 months ago
[CI] Broaden structured-output issue auto-labeling (#54645)
Flora Feng committed about 2 months ago
[Bugfix][PP] Never drop a decoding request from the sampled-token broadcast (#54436)
Ausar committed about 2 months ago
[XPU] [CI] Add retry for v1/sample in Intel GPU CI (#53669)
xiangdong committed about 2 months ago
Wentao Ye committed about 2 months ago
Wentao Ye committed about 2 months ago
[Kimi-K3][Perf] Make native CUDA AttnRes the SM100 default (#54261)
Yongye Zhu committed about 2 months ago
[Kernel][Gemma4] Prune Triton sliding-window tiles for multimodal prefixes (#53147)
mobicham committed about 2 months ago
[Bugfix][Spec Decode] Take the DFlash draft's RoPE layout from its own config (#54373)
Zihan Zhang committed about 2 months ago
[CI] Mark 1-GPU L4 test steps with device: l4 for EKS migration (#54549)
Kevin H. Luu committed about 2 months ago
[Perf][Kernel] Tune cooperative topk for medium batch-sizes (#53382)
Roberto L. Castro committed about 2 months ago