A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Bugfix][Core] Emit --no-{key} for false BooleanOptionalAction flags in YAML config (#51573)
Raj Vijay Firke committed 3 months ago
[Bugfix][Model] Fix Qwen3.5 MTP for text-only checkpoints (#50734)
efschu committed 3 months ago
[Doc] Add Crusoe Managed Inference deployment guide (#49353)
Emmanuel Acheampong committed 3 months ago
[CI][XPU] Add VLLM_DISABLE_COMPILE_CACHE=1 for other random failed cases in Intel GPU CI (#51604)
xiangdong committed 3 months ago
[V1][Scheduler] Apply Mamba alignment before encoder caps (#51603)
Jiangyun Zhu committed 3 months ago
[XPU] bump up xpu kernel to v0.1.12.3 (#50441)
Kunshang Ji committed 3 months ago
[CI] Upgrade huggingface-hub to 1.27.0 (#51422)
Andreas Karatzas committed 3 months ago
Fix Gemma 4 for upcoming Transformers version (#49797)
Harry Mellor committed 3 months ago
[KV Connector] Canonical CPU layout for parallelism-agnostic KV offload (#48414)
Itay Etelis committed 3 months ago
[ROCm][CI] Extend ROCm AITER MHA (FA) coverage (#40958)
Andreas Karatzas committed 3 months ago
[Bugfix] Fix lfm2 tool parser dropping calls with brackets or newline… (#48171)
Zetian Li - ikun committed 3 months ago
[EC Connector] Call to EC Connector update_connector_output from scheduler (#49579)
omerpaz95 committed 3 months ago
`[Model][Quantization] Add Ling-3.0-flash-fp8 support` (#51265)
zexplorerhj committed 3 months ago
Isotr0py committed 3 months ago
[CPU] Enable GPTQ and AWQ quantization for s390x (#51148)
Rehan Khan committed 3 months ago
[CPU] Restore linear dispatch for small unquantized GEMMs (#51379)
Li, Jiang committed 3 months ago
Add tiering offloading metrics (#48798)
Srinivas Krovvidi committed 3 months ago
[BugFix] Use file:// rendezvous for single-node executors to eliminate startup port races (#50999)
aoshen02 committed 3 months ago
[Bugfix][Parser] Emit REASONING_END for Inkling tool calls that follow no thinking block (#50528)
Jason committed 3 months ago
[Kernel][XPU] Tensor-descriptor operand loads for Triton W8A8 scaled_mm (#47205)
Lena Onyshchenko committed 3 months ago