A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 13:30 UTC
main
Better error message for missing mistral params.json (#17132)
Michael Goin committed over 1 year ago
[Misc] Add example to run DeepSeek with Ray Serve LLM (#17134)
Rui Qiao committed over 1 year ago
Add chat template for Llama 4 models (#16428)
Maximilien de Bayser committed over 1 year ago
Add collective_rpc to llm engine (#16999)
Yinghai Lu committed over 1 year ago
[Docs] Generate correct github links for decorated functions (#17125)
Russell Bryant committed over 1 year ago
Improve configs - `LoRAConfig` + `PromptAdapterConfig` (#16980)
Harry Mellor committed over 1 year ago
Add `:markdownhelp:` to `EngineArgs` docs so markdown docstrings render properly (#17124)
Harry Mellor committed over 1 year ago
Eyshika Agarwal committed over 1 year ago
existing torch installation pip command fix for docs (#17059)
Atilla committed over 1 year ago
Updating builkite job for IBM Power (#17111)
Aaruni Aggarwal committed over 1 year ago
[CI] Add automation for the `tool-calling` github label (#17118)
Russell Bryant committed over 1 year ago
[V1][Spec Decoding] Add num_drafts and num_accepted_tokens_per_position metrics (#16665)
Mark McLoughlin committed over 1 year ago
[Misc] refactor example series - structured outputs (#17040)
Reid committed over 1 year ago
Add missing rocm_skinny_gemms kernel test to CI (#17060)
Michael Goin committed over 1 year ago
[Frontend] Using matryoshka_dimensions control the allowed output dimensions. (#16970)
wang.yuqi committed over 1 year ago
Shanshan Shen committed over 1 year ago
Improve static type checking in `LoRAModelRunnerMixin` (#17104)
Harry Mellor committed over 1 year ago
[Misc] Remove OLMo2 config copy (#17066)
Isotr0py committed over 1 year ago
[V1][PP] Optimization: continue scheduling prefill chunks (#17080)
Rui Qiao committed over 1 year ago
Fix OOT registration test (#17099)
Harry Mellor committed over 1 year ago