A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Docs] Use incremental builds for C++ changes in `AGENTS.md` (#53098)
Thien Tran committed 2 months ago
[Bugfix][GDN] Reset speculative decode count for an empty draft schedule (#53077)
Kevin H. Luu committed 2 months ago
Harry Mellor committed 2 months ago
[3/N] Harden Transformers modelling backend multi-modal path (#51827)
Harry Mellor committed 2 months ago
[CI] replace shellcheck script with shellcheck-py hook (#52572)
Turner Jabbour committed 2 months ago
[BugFix][Mooncake] Fix Mooncake saves from sparse Mamba block tables (#51362)
Ziming Huang committed 2 months ago
[Rust Frontend] Fix Qwen parser auto-detection (#51169)
Sage committed 2 months ago
Eoin-Houstoun committed 2 months ago
[ROCm] Fix DeepSeek V4 indexer numerics and coverage (#50803)
Andreas Karatzas committed 2 months ago
[XPU][Tests] Make tests device-agnostic (#51968)
pmanczak committed 2 months ago
[Bugfix] Return HTTP 400 instead of 501 for unknown chat roles in DeepSeek encoders (#53071)
JC-ut0 committed 2 months ago
[Kernel][Kimi] fused vision q/k roper kernel (#50400)
rongfu.leng committed 2 months ago
[CI] Fix nonexistent dependency for data-parallel example test selection (#53026)
Taneem Ibrahim committed 2 months ago
[Bugfix][Security] Guard _load_ov2_processor with resolve_trust_remote_code (#52952)
Juan Pérez de Algaba committed 2 months ago
[Rust][Benchmark] Align speed-bench CLI flags with Python and add flag parity test (#51592)
Roy Wang committed 2 months ago
[Refactor] Remove InputPreprocessor (#53064)
Cyrus Leung committed 2 months ago
Anton A committed 2 months ago
[CI][Bugfix] Update distributed DP API server test path (#52939)
Kevin H. Luu committed 2 months ago
[Bugfix][Rust Frontend] Reject n > 1 in the `/inference/v1/generate` route (#52844)
Quentin Gallouédec committed 2 months ago
[Bugfix][EPD] Fix encoder round-robin fan-out (#52491)
Ankit Nakhawa committed 2 months ago