A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Bugfix][Reasoning] Nemotron V3: surface reasoning as content when thinking is unterminated (#39091)
Andrii Skliar committed 5 months ago
[Model] Remove obsolete ERNIE models (#45127)
Tiezhen WANG committed 5 months ago
Feature/offloading manager stats (#35669)
Srinivas Krovvidi committed 5 months ago
[Bugfix][Rust Frontend] Fix missing added tokens in hf/fastokens tokenizer (#44683)
Isotr0py committed 5 months ago
[Bugfix] Add deepseek_v32 to Quark dynamic MXFP4 model type check (#39498)
Shantipriya Parida committed 5 months ago
[Rust Frontend] [CI] Unify Rust artifact builds with setuptools-rust (#44981)
Bugen Zhao committed 5 months ago
[Security] Fix remote DoS via invalid recovered token reinjection (#44744)
Juan Pérez de Algaba committed 5 months ago
[BUGFIX][XPU] fix xpu `flash_attn_varlen_func` interface (#45110)
Kunshang Ji committed 5 months ago
[Refactor] Rename rocm_moe.py to rocm_moe_rdna.py (#45011)
JartX committed 5 months ago
[ROCm][CI] Defer AITER sampler import and isolate server test PYTHONPATH (#44823)
Andreas Karatzas committed 5 months ago
Harry Mellor committed 5 months ago
[SpecDecode] Reduce TP communication for large-vocab draft models speculative decoding (#39419)
yiheng committed 5 months ago
[ROCm][gpt-oss] Hybrid CDNA4 swizzle gate for A8W4 MoE (#44804)
xiaohuguo2023 committed 5 months ago
[Bench] Add BFCL dataset for vllm bench serve tool-calling workloads (#42457)
Li committed 5 months ago
Model/colbert autoweightsloader (#44999)
Furkan F committed 5 months ago
Change from owning configs to owning config utils (#45058)
Harry Mellor committed 5 months ago
Fix Harmony tool descriptions for optional fields (#44686)
Varun Vinayak Shenoy committed 5 months ago
[Rust Frontend] Add seed_oss and step3p5 reasoning parsers (#44552)
Yaoming Zhan committed 5 months ago
[Bugfix] Handle HWC images in ImageProcessorItems.get_image_size (#45057)
Akshat katiyar committed 5 months ago
[Refactor] Remove dead states from chat completion serving (#45081)
Flora Feng committed 5 months ago