A high-throughput and memory-efficient inference and serving engine for LLMs
main
[CI/Build] Give the Torch ABI audit time to start (#56596)
Kevin H. Luu committed about 1 month ago
[Build] Define _USE_MATH_DEFINES for FlashMLA targets (#54007)
Srijan Keshri committed about 1 month ago
[CI] Sample GPU utilization and memory alongside test timelines (#56541)
Kevin H. Luu committed about 1 month ago
[Frontend] create unified Cohere parser (#56392)
Walter Beller-Morales committed about 1 month ago
[ROCm][CI] Stage F gating (#55252)
Andreas Karatzas committed about 1 month ago
[DSV4.1] Remove compressor-aware image sentinel token padding (#56554)
Isotr0py committed about 1 month ago
[Bugfix][Examples] Launch prefill and decode concurrently in NixlPushConnector demo (#55088)
junuxyz committed about 1 month ago
[Bugfix][Spec Decode] Avoid fastsafetensors deadlock for PP draft models (#54416)
Canlin Guo committed about 1 month ago
[Rust Frontend] Support `generation` blocks in HF chat template (#56378)
Bugen Zhao committed about 1 month ago
[3/N] HiSparse: host-resident sparse-MLA decode hot-buffering (#53781)
Matthew Bonanni committed about 1 month ago
[5/N][warmup][DSv4] Migrate NVIDIA CuTeDSL attention kernels (#53566)
Roberto L. Castro committed about 1 month ago
[Rust Frontend] Honor HF revisions, offline mode, and cache directory (#56386)
Bugen Zhao committed about 1 month ago
Revert "[Agents] Link Triton skill to JIT kernel warmup guide" (#56546)
Woosuk Kwon committed about 1 month ago
[CI] Bump CUTLASS DSL to 4.7 (#54927)
Jiangyun Zhu committed about 1 month ago
[Rust Frontend][Multimodal] Accept preprocessed multimodal gRPC features (#55047)
Biswa Panda committed about 1 month ago
[XPU][CI] skip DeepSeek-V4.1-Flash in `test_tensor_schema.py` (#56463)
liuzhenwei committed about 1 month ago
Divakar Verma committed about 1 month ago
[CI] Give every Buildkite step an explicit key, and a hook to enforce it (#56329)
Tahsin Tunan committed about 1 month ago
[Bugfix][NIXL] Don't evict a remote engine a transfer is still reading from (#54689)
Yizheng Jiao committed about 1 month ago
[ROCm][DSV4.1][Perf] Use AITER mHC for the delayed pre block (#56503)
Fangzhou Ai committed about 1 month ago