A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Frontend] Migrate Responses harmony input validation to VLLMValidationError (#55701)
Adababy committed about 2 months ago
[Bugfix][Spec Decode] Resolve n_predict from text_config for Qwen3.5 multimodal MTP (#55369)
Soumyajit Ghosh committed about 2 months ago
[Frontend] Add stateless /v1/responses/render endpoint (#50195)
Francisco Javier Arceo committed about 2 months ago
[Bugfix][Quantization][XPU] Fix moe_wna16 linear weight loading (#52651)
Artur Fierka committed about 2 months ago
[Bugfix] OffloadingConnector: stop zeroing offload hits under MTP/EAGLE spec decode (#52771)
Kam Basra committed about 2 months ago
[Qwen3.8-Flash-Next] Support FP8 indexer cache for QSA (#54890)
Thien Tran committed about 2 months ago
[Docs] Add OLMo 2 to batch-invariance tested models (#55691)
Sean Westfall committed about 2 months ago
[XPU][LoRA] Support LoRA for DeepSeek V4 on XPU (#53689)
Chaojun Zhang committed about 2 months ago
[XPU] Route grouped_topk to the fused _moe_C kernel on XPU (#53580)
Marceli Fylcek committed about 2 months ago
[Frontend] Migrate Responses API validation errors to VLLMValidationError (#50257)
Adababy committed about 2 months ago
[ROCm][CI] Add attention-sink support to ROCm AITER sparse MLA (#54404)
Andreas Karatzas committed about 2 months ago
[CI] [Test] skip test_wna16_cuda_high_bit_skips_humming on non-CUDA platforms (#55660)
Chaojun Zhang committed about 2 months ago
[KVConnector] Guard lmcache_mp_connector state transition with num_external_tokens (#47505)
AlexHuang committed about 2 months ago
[Bugfix] DSv4 MXFP4 selector: stop narrowing explicit aliases to their BF16 variant (#53586)
Gabriel Wu committed about 2 months ago
[Bugfix][MooncakeStore] Fix finish-time save crash on hybrid models (#54643)
Zhewen Li committed about 2 months ago
[CI][ROCm] Temporarily skip unsupported HY-V4 initialization (#55653)
Andreas Karatzas committed about 2 months ago
[EPD] Add ECMooncakeConnector for encoder cache over Mooncake TransferEngine (#41567)
Teng Ma committed about 2 months ago
[Bugfix][Audio] Restore soundfile-first automatic decoding (#55642)
Andreas Karatzas committed about 2 months ago
[XPU] Use fused_input_norm kernel in FusedInputNorm (#52945)
zofia committed about 2 months ago
[Qwen3.8-Flash-Next] Remove torch.compile for NVIDIA implementation (#55272)
Thien Tran committed about 2 months ago