A high-throughput and memory-efficient inference and serving engine for LLMs
main
[XPU][CI]Refine .buildkite/ci_config_intel.yaml for Intel GPU CI (#46674)
xiangdong committed 4 months ago
[ROCm][ [Perf] sparse attention optimization on minimax-m3 (#46546)
Hongxia Yang committed 4 months ago
[CPU][RISC-V] Add RVV path for W4A8 INT4 GEMM (#45269)
wcy committed 4 months ago
[Model] Remove AquilaForCausalLM, AquilaModel (#46605)
Tiezhen WANG committed 4 months ago
[CPU][Spec Decode] Enable DFlash SD for CPU (#44029)
guybd committed 4 months ago
[Rust Frontend] Forward `VLLM_ENGINE_READY_TIMEOUT_S` via `--args-json` (#44610)
Kai committed 4 months ago
[Rust Frontend] Make `ToolParserOutput` a seq of `ToolParserEvent` to preserve order (#46584)
Bugen Zhao committed 4 months ago
[ROCm][Perf] Tune wvSplitK on gfx1151 (#40784)
Matthias Gehre committed 4 months ago
[SimpleCPUOffloadConnector] Fix remaining global→block conversions under PCP/DCP (#46394)
Jeff (Junze) Ma committed 4 months ago
[Perf] Enable + tune FlashInfer fused allreduce at world_size=16 on SM 10.3 (GB300) (#46392)
Jeff (Junze) Ma committed 4 months ago
[Hardware][AMD][CI] Move Metrics, Tracing (2 GPUs) & make optional (#46686)
Matt committed 4 months ago
[ROCm][CI] Expand basic correctness target suites (#46573)
Andreas Karatzas committed 4 months ago
[ROCm][Bugfix] Fix chunk alignment when using context parallelism with TRITON_MLA (#46114)
Micah Williamson committed 4 months ago
[CPU] Enable chunked prefill and prefix caching for qwen3.5 (#46202)
Tianmu Li committed 4 months ago
[AMD][CI] Fix Pipeline + Context Parallelism test group (#46650)
Aarushi Jain committed 4 months ago
[Rust Frontend] Introduce unified parser interface & combined parser (#46583)
Bugen Zhao committed 4 months ago
[Bugfix] fix: stream Mimimax m2 tool call string arguments (#46382)
Chauncey committed 4 months ago
[ROCm] Remove erroneous inclusion of gptq_marlin as supported quant scheme on ROCm (#46655)
Micah Williamson committed 4 months ago
[Kernel] Add swap AB optimization to fused_moe_kernel (#36559)
Xin Yang committed 4 months ago
[Perf][KVConnector][Mooncake] Parallelize KV load with a receive-thread pool (#45971)
Yifan Qiao committed 4 months ago