A high-throughput and memory-efficient inference and serving engine for LLMs
main
Sergey Shlyapnikov committed 21 days ago
[ROCm][Perf][GLM-5.3-Flash] "Fit kpool top-k indices to AITER" with a single Triton kernel (#58008)
Simon Danielsson committed 21 days ago
[Refactor] Share workspace and model runner init between GPU and XPU workers (#59200)
aoshen02 committed 21 days ago
[Security] Bump pyjwt and rand for remaining GHSAs (#59427)
Juan Pérez de Algaba committed 21 days ago
[Feat] Support PP with PCP in GPU Model Runner V2 (#59139)
Qiu Chunshuo committed 21 days ago
[Bugfix][Core] Schedule encoder-only prompts larger than one step (#59029)
Giulio De Pasquale committed 21 days ago
[Test] Skip IPC weight-transfer test on XPU platforms (#59379)
Chaojun Zhang committed 21 days ago
[CI/Build] Fix pre-commit (#59426)
Cyrus Leung committed 21 days ago
[KV-Offloading][TP] : Expand replicated_layout detection to multi-group MLA (#57652)
Varun Sundar Rabindranath committed 21 days ago
[Feature][Rust Frontend] Add Shutdown control RPC (#59316)
JulienDarve committed 21 days ago
[Docs] Reinstate docs build gate for PRs (#59411)
Harry Mellor committed 21 days ago
[MM][Mistral] Add compile support for Pixtral vision encoders (#57168)
Jakub Zakrzewski committed 21 days ago
[CPU] Conv1d optimised kernel for aarch64 (#54093)
almayne committed 21 days ago
[XPU] Dispatch nn.LayerNorm to fused SYCL kernel via CustomOp (#57172)
Michał Ganczarenko committed 21 days ago
[Security] Bump remaining Dependabot packages (excl. ignored) (#59315)
Juan Pérez de Algaba committed 21 days ago
[Security] Authenticate shared-memory multimodal cache handles (#59357)
Clinton Thomas committed 21 days ago
[Hardware][Power] Enable W4A16 (AWQ & GPTQ) quantization on POWER10 using VSX (#59149)
Rukhaiya2004 committed 21 days ago
docs: add gemma-2, SmolLM2, Qwen2.5-Coder, Llama-3.2-1B to batch invariance tested models (#54441)
yuvalluria committed 21 days ago
[MM] Keep device input normalization fused when encoder compilation is enabled (#59195)
Jakub Zakrzewski committed 21 days ago
[MyPy][3/N] Fix MyPy errors in test groups (part 3) (#55939)
Taimys committed 21 days ago