A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Perf][1/N] Expand Triton kernel warmup coverage, DSv4 (#46634)
Roberto L. Castro committed 4 months ago
Bump flashinfer version to 0.6.13 (#46683)
Wei Zhao committed 4 months ago
Add MiniMax-M3 modelopt nvfp4 support (#46756)
Jason Li committed 4 months ago
[FS-Offloading] Batch Lookup in C (#46713)
Varun Sundar Rabindranath committed 4 months ago
[Bugfix][Quantization] Fix W8A8 int-quantized scheme selection regression (#46860)
HDCharles committed 4 months ago
[MyPy] Fix mypy incompatible assignment errors in LRUCacheLoRAModelManager (#44657)
Martin Hickey committed 4 months ago
[BugFix] Revert "[KV Offload] Use background thread for mmap / cpu_tensors pinning" (#46958)
Varun Sundar Rabindranath committed 4 months ago
[Kernel] Triton MLA logits workspace (#46819)
Nicolò Lucchesi committed 4 months ago
[Kernel][Helion][1/N] Add Helion kernel for fused_qk_norm_rope (#44010)
Xiaohong (Sean) Chen committed 4 months ago
[Bugfix] Capture final-layer aux hidden state in deepseek_v2 backbone (#46973)
Michael Goin committed 4 months ago
[Rust Frontend] Keep literal "null" string for string-typed tool params (#46827)
Blas Rodriguez Irizar committed 4 months ago
[MoE Refactor] Standardize Humming MoE experts + utilities (#43373)
bnellnm committed 4 months ago
[XPU] exclude unsupported models for test_tensor_sechma.py (#47008)
Yan Ma committed 4 months ago
[Frontend] Consolidate scale out entrypoints (#44512)
wang.yuqi committed 4 months ago
[Perf][2/N] Expand Triton kernel warmup coverage, Qwen (#46750)
Roberto L. Castro committed 4 months ago
[mypy] Enable mypy for tests directory (#47018)
Martin Hickey committed 4 months ago
Fix model info cache for package models (#46567)
soaringk committed 4 months ago
[Bugfix] Use native SiLU activation in CPU fused MoE (#45961)
Alden Lobo committed 4 months ago
[Bugfix] Transformers backend: apply learned lm_head.bias for tied-embedding models (#46835)
John committed 4 months ago
[CI Failure] Add transformers version check for openai/privacy-filter (#47011)
wang.yuqi committed 4 months ago