A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 17:15 UTC
main
Fix non-contiguous input passed to Marlin kernel (#15319)
Qubitium-ModelCloud committed over 1 year ago
[Fix] [torch.compile] Improve UUID system for custom passes (#15249)
Luka Govedič committed over 1 year ago
[V1] Enable V1 Fp8 cache for FA3 in the oracle (#15191)
Lucas Wilkinson committed over 1 year ago
[Misc][Doc] Add note regarding loading `generation_config` by default (#15281)
Roger Wang committed over 1 year ago
[Frontend] Support tool calling and reasoning parser (#14511)
Robin committed over 1 year ago
[V1][Spec Decode] Use better defaults for N-gram (#15358)
Woosuk Kwon committed over 1 year ago
[V1][Spec Decode] Respect prompt_lookup_max (#15348)
Woosuk Kwon committed over 1 year ago
[Bugfix] consider related env vars for torch.compiled cache hash (#14953)
DefTruth committed over 1 year ago
[Misc] Add tuned R1 w8a8 and MoE configs for NVIDIA L20 (#15322)
DefTruth committed over 1 year ago
[ci/build] fix broken tests in LLM.collective_rpc (#15350)
youkaichao committed over 1 year ago
[ci/build] update torch nightly version for GH200 (#15135)
youkaichao committed over 1 year ago
[V1][Usage] Refactor speculative decoding configuration and tests (#14434)
shangmingc committed over 1 year ago
Fix v1 supported oracle for worker-cls and worker-extension-cls (#15324)
hijkzzz committed over 1 year ago
[doc] Add back previous news (#15331)
Chen Zhang committed over 1 year ago
Remove openvino support in favor of external plugin (#15339)
Russell Bryant committed over 1 year ago
[BugFix][Typing] Fix Imprecise Type Annotations (#15208)
Wang Ran (汪然) committed over 1 year ago
[V1] Add `disable-any-whitespace` option support for xgrammar (#15316)
Russell Bryant committed over 1 year ago
[Model] Support Tele-FLM Model (#15023)
Naitong Yu committed over 1 year ago
[Bugfix] LoRA V0 - Fix case where `max_num_seqs` is between cudagraph capture sizes (#15308)
Varun Sundar Rabindranath committed over 1 year ago
[Bugfix] Fix torch.compile raise FileNotFoundError (#15278)
Jee Jee Li committed over 1 year ago