A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 02:00 UTC
main
Hongxia Yang committed almost 2 years ago
[BUGFIX] When skip_tokenize_init and multistep are set, execution crashes (#12277)
Aleksandr Malyshev committed almost 2 years ago
[bugfix] moe tuning. rm is_navi() (#12273)
Divakar Verma committed almost 2 years ago
[Core] Free CPU pinned memory on environment cleanup (#10477)
Jani Monoses committed almost 2 years ago
[Misc] Set default backend to SDPA for get_vit_attn_backend (#12235)
wangxiyuan committed almost 2 years ago
[Misc] Remove experimental dep from tracing.py (#12007)
Adrian Cole committed almost 2 years ago
[Bugfix] Multi-sequence broken (#11898)
Andy Lo committed almost 2 years ago
[v1][stats][1/n] Add RequestStatsUpdate and RequestStats types (#10907)
Ricky Xu committed almost 2 years ago
[Kernel] fix moe_align_block_size error condition (#12239)
Jinzhen Lin committed almost 2 years ago
[Bugfix] fix race condition that leads to wrong order of token returned (#10802)
Jannis Schönleber committed almost 2 years ago
[ci/build] update nightly torch for gh200 test (#12270)
youkaichao committed almost 2 years ago
[Platform] improve platforms getattr (#12264)
Mengqing Cao committed almost 2 years ago
Remove pytorch comments for outlines + compressed-tensors (#12260)
Thomas Parnell committed almost 2 years ago
[V1][Bugfix] Fix data item ordering in mixed-modality inference (#12259)
Roger Wang committed almost 2 years ago
[torch.compile] transparent compilation with more logging (#12246)
youkaichao committed almost 2 years ago
[Bugfix] Fix mm_limits access for merged multi-modal processor (#12252)
Cyrus Leung committed almost 2 years ago
[Misc] Remove redundant TypeVar from base model (#12248)
Cyrus Leung committed almost 2 years ago
[Misc]Add BNB quantization for PaliGemmaForConditionalGeneration (#12237)
Jee Jee Li committed almost 2 years ago
[Misc] Rename `MultiModalInputsV2 -> MultiModalInputs` (#12244)
Cyrus Leung committed almost 2 years ago
[ci/build] disable failed and flaky tests (#12240)
youkaichao committed almost 2 years ago