A high-throughput and memory-efficient inference and serving engine for LLMs
main
[CI] Add GSM8K accuracy test for amd/DeepSeek-V4-Flash-MXFP4 (#50632)
Colin Z committed about 2 months ago
[XPU] update key supported models (#53494)
Yan Ma committed about 2 months ago
[K3 Perf] Optimize k3 mamba metadata preparation, 6.6~7.6x kernel performance improvement (#52388)
Wentao Ye committed about 2 months ago
[Feature][DSpark]: Logprobs adaptive verification (#52242)
Naveenraj Kamalakannan committed 2 months ago
[Quantization][Humming] Support MXFP4 weight + block-FP8 activation for MoE (#51332)
Elvir Crnčević committed 2 months ago
[Bugfix] Handle parenthesized Gemma4 tool calls (#53657)
Taneem Ibrahim committed 2 months ago
[Pooling UX] Improve serve --task error guidance (#53467)
Taneem Ibrahim committed 2 months ago
[Profiler] Fix start_profile permanently no-op after max_iterations auto-stop (#51839)
Elvir Crnčević committed 2 months ago
[ROCm][CI] Skip ModernBERT FP8 MTEB test when no FP8 ScaledMM kernel exists (#53589)
djramic committed 2 months ago
[Config][EC] Normalize producer-only encoder config (#53656)
Tianyu Guo committed 2 months ago
[Bugfix][DeepSeek V4] Handle trailing system messages in prompt rendering (#51262)
Jiahao Liang committed 2 months ago
[sleep functionality] code refactor about sleep/wake_up (#50431)
Ning Xie committed 2 months ago
[Agents] Add kernel microbenchmark skill (#53688)
Thien Tran committed 2 months ago
[Bugfix][MM] Fix JinaVL processing cache order (#53553)
Jiatai Wang committed 2 months ago
[Frontend] Move cli_args.py and dp_supervisor.py out openai folder (#53659)
wang.yuqi committed 2 months ago
[Bugfix][MRV2] Dispatch uniform decode to a padded FULL cudagraph (#53407)
xiaohuguo2023 committed 2 months ago
Vllm recipes tool improve (#53325)
Louie Tsai committed 2 months ago
[Model] Migrate FlexOlmo, Olmo3 and Hunyuan V1/VL to the Transformers modeling backend (#53615)
Harry Mellor committed 2 months ago
Seunghyuk Park (shepark) committed 2 months ago
Karan Shelar committed 2 months ago