A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 8, 05:00 UTC
main
[Model Runner V2] support bad_words sampling param (#33433)
zhrrr committed 9 months ago
[NemotronH] Do not force router to run in fp32 (#34582)
roikoren755 committed 9 months ago
Targeting the MI355 agent pool with all existing tests (#34629)
Alexei-V-Ivanov-AMD committed 9 months ago
Nicolò Lucchesi committed 9 months ago
[Bugfix] Treat generation_config max_tokens as default not ceiling (#34063)
Almog Tavor committed 9 months ago
[CI] Enable mypy coverage for individual excluded files (#34292)
Lucas Kabela committed 9 months ago
(bugfix): Fixed encode in LLM entrypoint for IOProcessr plugin prompts (#34618)
Christian Pinto committed 9 months ago
[ROCm][CI] Fix plugins test group; updating terratorch and dependencies (#34589)
Andreas Karatzas committed 9 months ago
[Bugfix] Fix ARC touch KeyError for non-ready T1 blocks in kv offload (#34576)
Yiqi Xue committed 9 months ago
Fix call to moe_mk in modelopt MoE modules (required for LoRA) (#34575)
danisereb committed 9 months ago
[Models] Fuse Qwen3.5 GDN's qkvz_proj and ba_proj (#34492)
Isotr0py committed 9 months ago
[CI] Disable precompiled wheel path in CI image builds (#34606)
Amr Mahdi committed 9 months ago
Bump `lm-eval` version for Transformers v5 compatibility (#33994)
Harry Mellor committed 9 months ago
[CI][Tracing] Fix race condition by adding server readiness check (#34364)
emricksini-h committed 9 months ago
[Scheduler][ASR] Fix CrossAttn blocks per-request for Variable length encoder inputs (#31058)
Ekagra Ranjan committed 9 months ago
[CI][Metrics] Stabilize tests with polling and subprocess guards (#34566)
Andreas Karatzas committed 9 months ago
[Bugfix] Fix Dynamo unexpected keyword argument (#34320)
Samu Tamminen committed 9 months ago
Revert "[Misc] fix qwen3.5 config" (#34610)
Roger Wang committed 9 months ago
[Misc] fix qwen3.5 config (#34604)
JJJYmmm committed 9 months ago
[Renderer] Move InputPreprocessor into Renderer (1.5/2) (#34598)
Cyrus Leung committed 9 months ago