A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Test][Determinism] Cover VLM batch invariance in default execution mode (#56528)
Shenglei Fu committed about 1 month ago
[LoRA] Add LoRA support for DeepSeek-V4 Flash Vision (#55897)
linitra24 committed about 1 month ago
[XPU][CI] fix jit_warmup_triton_launcher (#56670)
liuzhenwei committed about 1 month ago
[Pooling] Report actual input token usage for scoring APIs (#56573)
Taneem Ibrahim committed about 1 month ago
[Core] Fix ValueError on KV load failure with a hybrid KV cache (#50388)
Kebe committed about 1 month ago
[Bugfix][EPD] Preserve explicit multimodal UUIDs with caches disabled (#56432)
Tianyu Guo committed about 1 month ago
Juntian Liu committed about 1 month ago
[Bugfix] Avoid repeated dummy initialization and random CPU Engram fills (#56682)
Woosuk Kwon committed about 1 month ago
[Bugfix][Gemma4] Keep image kwargs out of video preprocessing (#56652)
Ylean committed about 1 month ago
[Perf][Kernel] Integrate DeepSelect TopK for the DSA sparse indexer (#56464)
Jiangyun Zhu committed about 1 month ago
[CI] Fix NIXL transfer-rank geometry fixture (#56672)
Lucas Wilkinson committed about 1 month ago
[Bugfix][KV Offload] Submit CPU stores on no-forward steps (#56621)
Lucas Wilkinson committed about 1 month ago
[Pooling] Support prompt embeddings in MRV2 decoder pooling (#56639)
Taneem Ibrahim committed about 1 month ago
[Bugfix][Benchmark] Make streaming TTFT/E2E latency accounting consistent across endpoints (#55508)
Suraj committed about 1 month ago
[MRV2] Revert explicit Triton JIT warmup migration (#56654)
Woosuk Kwon committed about 1 month ago
[ROCm] Auto-enable breakable CUDA graphs for DeepseekV41ForCausalLM (#56349)
Markus Hartikainen committed about 1 month ago
[Kernel][Perf][Quantization] Fix odd-row performance cliff in per-token-group quantization (#56478)
Ayushman Singh committed about 1 month ago
[Misc] Clean up fast loader daemon quant method verification (#55656)
Isotr0py committed about 1 month ago
[9/N][warmup][DSv4] Migrate MHC TileLang kernels (#50178)
Roberto L. Castro committed about 1 month ago
[6/N][warmup][DSv4] Migrate sampling, and DFlash JIT kernels (#56323)
Roberto L. Castro committed about 1 month ago