A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Bugfix] Disable sequence parallelism for Dots3 NOTE (#52172)
范裕达 committed 2 months ago
[LoRA][Gemma4] Support vision tower LoRA (#42662)
linitra24 committed 2 months ago
[Model] [Quantization] Add Ling hybrid MXFP4 routed experts support (#52114)
zexplorerhj committed 2 months ago
[Frontend] Log output token IDs at DEBUG level (#52098)
yang rui committed 2 months ago
[Misc] Add missing return type annotations in outputs.py (#52145)
Vineeta Tiwari committed 2 months ago
[XPU] Add tuned Mamba SSU configs for Intel Arc Pro B70 (#50534)
pmanczak committed 2 months ago
[Bugfix] Preserve Anthropic disable_parallel_tool_use (#52021)
Taneem Ibrahim committed 2 months ago
chore: Upstream Cohere parser fixes + tests (#51998)
JasonCohere committed 2 months ago
[Doc] Fix stale rejection_sample_method and synthetic_acceptance_rate (#51611)
QWERQWERQWE86 committed 2 months ago
fix(security): enforce audio decode duration limit in NanoNemotronVL (#50221)
Juan Pérez de Algaba committed 2 months ago
Hardware-agnostic model definition via HF transformer backend (1/N) (#49458)
Thomas Ortner committed 2 months ago
[Docs] Fix `WhisperEncoderLayer.forward` docstring in `dots3_note` (#52134)
Harry Mellor committed 2 months ago
[CI/Build][CPU] Shrink triton-cpu-build layer by dropping build artifacts (#52127)
Li, Jiang committed 2 months ago
[ROCm][Perf] Kimi-K3 Remove prefill pipeline stall in chunk KDA (#51862)
kliuae committed 2 months ago
[Feature] Mask Replay (#49577)
vx120 committed 2 months ago
Cyrus Leung committed 2 months ago
[Core] Configure custom encoder cache managers from VllmConfig (#51251)
hotTea committed 2 months ago
[Bugfix][R3] Size monolithic routing replay buffer for DP (#50874)
TomerBN-Nvidia committed 2 months ago
[CPU] Ship triton-cpu wheel and fix several hardcoded pin_memory=True (#52092)
Li, Jiang committed 2 months ago
[Model] Skip unused Jina V5 output layers (#52037)
kiroxu committed 2 months ago