A high-throughput and memory-efficient inference and serving engine for LLMs
main
[XPU] Route block-quantized FP8 weights to the W8A8 kernel (#50787)
Chaojun Zhang committed 2 months ago
[Bugfix][XPU] Run GDN attention as eager break under breakable cudagraph (#51928)
Huanxing committed 2 months ago
[Model] Support R3 capture with DeepGEMM MegaMoE (#51831)
aoshen02 committed 2 months ago
[Attention] Move context_lens_tensor compute into GDN prefill path (#51913)
Xin Yang committed 2 months ago
[CI/Release][XPU] fix workdir path for triton shim job (#51923)
Kunshang Ji committed 2 months ago
[Bugfix][Quantization] Reuse online NVFP4 MoE kernel across reloads (#50074)
Matej Sirovatka committed 2 months ago
[Refactor][MRV2] Unify uniform decode token count helper (#51917)
Lucas Wilkinson committed 2 months ago
[Bugfix] Avoid repeated layerwise reload warning scans (#49505)
aoshen02 committed 2 months ago
[Frontend] Make `model` optional on all `/derender` request classes (#51463)
Vinay R Damodaran committed 2 months ago
[XPU][CI]Change to use global VLLM_DISABLE_COMPILE_CACHE=1 in Intel GPU CI (#51905)
xiangdong committed 2 months ago
[Bugfix][V1][Multimodal] Recover from P0/P1 processor cache drift (#46747) (#46747)
WillZZZy committed 2 months ago
[CI/Release][1/N][XPU] Publish XPU Triton shim index (#51759)
Kunshang Ji committed 2 months ago
[ROCm][Perf] Kimi-K3 Fused kernel for KDA decode (#50654)
kliuae committed 2 months ago
[Bugfix][Triton] Make fp8_min/fp8_max constexpr in _quantize_pad_fp8_kernel (#51872)
Max Hu committed 2 months ago
[CI/Build] Use file rendezvous for local distributed tests (#51652)
Stellar鱼 committed 2 months ago
[Hardware][AMD] Enable fused bf16→fp32 router GEMM on ROCm (#50268)
Matvei Pashkovskii committed 2 months ago
[Perf] Avoid more GPU<->CPU syncs on the model execution path (#51738)
Nick Hill committed 2 months ago
[CI] Stabilize tensor IPC multiprocessing tests (#50804)
Andreas Karatzas committed 2 months ago
[Bugfix][TieredOffloading] : Return HIT_PENDING when KV promotion is triggered (#51840)
Varun Sundar Rabindranath committed 2 months ago
[Docs][RL] Rewrite weight-transfer docs; standardize examples (#51729)
Aaron Hao committed 2 months ago