A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Frontend] Remove the slow tokenizer mode (#58545)
Harry Mellor committed 28 days ago
[Bugfix] Resolve the Hub revision once per repo (#56092)
Lucain committed 28 days ago
fix(config): apply presence_penalty/frequency_penalty from override-generation-config (#50769)
hcl committed 28 days ago
[Perf][Attention] Bound FlashInfer prefill dequantization scratch (#57918)
Yueh-Ting (eop) Chen committed 28 days ago
[Refactor] Remove dead or duplicate tests (#58446)
Wentao Ye committed 28 days ago
[Chore] Use Transformers v5 names and drop redundant processor `use_fast` (#58550)
Harry Mellor committed 28 days ago
[Bugfix][Pooling] Fix JinaVL label configuration and restore multimodal tests (#57347)
Linze Shi committed 28 days ago
Remove `.gemini/` and `CLAUDE.md` (#58541)
Harry Mellor committed 28 days ago
Fangzhou Ai committed 28 days ago
[XPU] enable XPU GRAPH by default (#51600)
liuzhenwei committed 28 days ago
[Multimodal] Reuse the supplied tokenizer in the MiniMax-M3 VL processor (#58460)
Zijing Liu committed 28 days ago
[Perf][Frontend] Offload streaming derender detokenization (#57528)
SHREY GAJJAR committed 28 days ago
[CPU] Gate the AVX10.2 paths on compiler support (#58133)
Ganesh R committed 28 days ago
Ziyang Ma committed 28 days ago
[Bugfix][Rust Frontend] Prevent MM timing from enabling debug tracing (#58378)
Reid committed 28 days ago
[MRV2] Validate MRV2 entrypoint logits processors (#57728)
Taneem Ibrahim committed 28 days ago
[Attention][CPU] Run Zen CPU encoder attention on zentorch SDPA (#54508)
Priyansh Jain committed 28 days ago
[Bugfix][XPU] store the pointer raw bit pattern instead of its numeric value (#54514)
Yejing Lai committed 28 days ago
[Bugfix][CI] Fix the flaky sharded-sampling tests, and the engine teardown need (#58342)
stefankoncarevic committed 28 days ago
[Bugfix] Capture prefill kernels for mixed FULL graphs (#58275)
Andreas Karatzas committed 28 days ago