A high-throughput and memory-efficient inference and serving engine for LLMs
main
[ROCm][CI] Transformers: pass only one of input_ids/inputs_embeds (#48258)
stefankoncarevic committed 4 months ago
[EC Connector] CPU Offloading EC Connector (#47423)
omerpaz95 committed 4 months ago
[Misc] Improve Matryoshka pooling dimensions validation (#48057)
Taneem Ibrahim committed 4 months ago
[BugFix][ModelRunner V2] Fix stale attn metadata in speculator prefill cudagraph capture (#48261)
Nick Hill committed 4 months ago
[XPU][CI]Adjust timeout_in_minutes in Intel GPU CI (#48418)
xiangdong committed 4 months ago
remove force channels_last in Idefics3MultiModalProcessor (#48467)
Yan Ma committed 4 months ago
lower memory required for capturing cudagraphs for large cudagraph sizes (#48483)
Omer Ullman Argov committed 4 months ago
[Mypy Fix] Split mypy work (#48490)
Wentao Ye committed 4 months ago
[CI] Add SPDX license header to Rust/Protobuf sources (#48472)
Bugen Zhao committed 4 months ago
[Core] Support fp32 lm_head for generation models via head_dtype (RFC #48305 §3.6) (#48390)
Karthik Kothuri committed 4 months ago
[Bugfix][KV Cache] Don't route uniform-page-size MLA+SWA models into DeepseekV4 packing (#48256)
Nicolò Lucchesi committed 4 months ago
[Attention] Make sliding-window support an explicit backend capability (#48011)
Nicolò Lucchesi committed 4 months ago
[Distributed][Perf] Enable FlashInfer MNNVL allreduce RMS quant fusion (#48064)
Mohammad Miadh Angkad committed 4 months ago
[ROCm][CI] Cache Rust builds by source inputs (#46527)
Andreas Karatzas committed 4 months ago
[BugFix] Restore full tokens for Qwen MTP When MoE SP (#48429)
Canlin Guo committed 4 months ago
[CPU] Create Proper Numa topology for s390x (#40714)
Rehan Khan committed 4 months ago
[Bugfix][UT]Fix EagleMiniCPMForCausalLM meet TypeError (#48452)
Yejing Lai committed 4 months ago
[CPU][Spec Decode] Support DFlash speculative decoding for GDN models on CPU (#46090)
guybd committed 4 months ago
Re-disable CUDA graph memory profiling on ROCm (#48440)
Rohan Potdar committed 4 months ago
[Bugfix][ROCm] Keep TP all_gather on base-class collective (#48446)
Fangzhou Ai committed 4 months ago