A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 12:15 UTC
main
[Core] Remove legacy input mapper/processor from V0 (#15686)
Cyrus Leung committed over 1 year ago
[Minor][Models] Pass partial_rotary_factor parameter to rope (#17266)
Wanrui Dai committed over 1 year ago
[Misc] Validate `stop_token_ids` contents (#17268)
Nick Hill committed over 1 year ago
Reid committed over 1 year ago
[Doc] Fix wrong github link in LMCache examples (#17274)
Kuntai Du committed over 1 year ago
[Bugfix] Fix Mistral3 spatial merge error (#17270)
Michael Goin committed over 1 year ago
[Bugfix] Fix cutlass dispatch for fp8/int8 to properly invoke M<=16 c… (#16751)
TherLF committed over 1 year ago
[Bugfix] Fix missing ARG in Dockerfile for arm64 platforms (#17261)
Lennart K. M. Schulz committed over 1 year ago
[BugFix] Fix vllm_flash_attn install issues (#17267)
Lucas Wilkinson committed over 1 year ago
[V1][Spec Decode] Make eagle compatible with prefix caching. (#17137)
Lily Liu committed over 1 year ago
[Metrics] Fix minor inconsistencies in bucket progression (#17262)
Cyrus Leung committed over 1 year ago
[doc] Add feature status legend (#17257)
Reid committed over 1 year ago
[Feature] support sequence parallelism using compilation pass (#16155)
cascade committed over 1 year ago
[NVIDIA] Support Cutlass MLA for Blackwell GPUs (#16032)
Kaixi Hou committed over 1 year ago
[Bugfix] Fix Lora Name Parsing (#17196)
Alex Brooks committed over 1 year ago
Flex Wang committed over 1 year ago
[MISC] Use string annotation types for class definitions (#17244)
Jade Zheng committed over 1 year ago
[Bugfix] Get a specific type of layer from forward context (#17222)
Chen Zhang committed over 1 year ago
[Bugfix] Fix QWen2 VL multimodal mapping (#17240)
Jee Jee Li committed over 1 year ago
[Misc] Make cached tokenizer pickle-compatible (#17048)
Cyrus Leung committed over 1 year ago