A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 09:30 UTC
main
[ci/Build] Skip Chameleon for transformers 4.46.0 on broadcast test #9675 (#9676)
Kevin H. Luu committed about 2 years ago
[Bugfix] Remove xformers requirement for Pixtral (#9597)
Michael Goin committed about 2 years ago
[Performance][Kernel] Fused_moe Performance Improvement (#9384)
Charlie Fu committed about 2 years ago
[Log][Bugfix] Fix default value check for `image_url.detail` (#9663)
Michael Goin committed about 2 years ago
[Model] Compute Llava Next Max Tokens / Dummy Data From Gridpoints (#9650)
Alex Brooks committed about 2 years ago
[CI/Build] Fix VLM test failures when using transformers v4.46 (#9666)
Cyrus Leung committed about 2 years ago
[torch.compile] Adding torch compile annotations to some models (#9641)
Yongzao committed about 2 years ago
Increase operation per run limit for "Close inactive issues and PRs" workflow (#9661)
Harry Mellor committed about 2 years ago
[Bugfix]Disable the post_norm layer of the vision encoder for LLaVA models (#9653)
litianjian committed about 2 years ago
[Doc] Move additional tips/notes to the top (#9647)
Cyrus Leung committed about 2 years ago
[torch.compile] expanding support and fix allgather compilation (#9637)
Yongzao committed about 2 years ago
[Kernel] add kernel for FATReLU (#9610)
Jee Jee Li committed about 2 years ago
[torch.compile] Adding torch compile annotations to some models (#9639)
Yongzao committed about 2 years ago
[core] simplify seq group code (#9569)
youkaichao committed about 2 years ago
[V1][Bugfix] Clean up requests when aborted (#9629)
Woosuk Kwon committed about 2 years ago
[Bugfix] Fix PP for ChatGLM and Molmo (#9422)
Cyrus Leung committed about 2 years ago
[XPU] avoid triton import for xpu (#9440)
Yan Ma committed about 2 years ago
[Bugfix]: Make chat content text allow type content (#9358)
Vinay R Damodaran committed about 2 years ago
[Bugfix] Use "vision_model" prefix for MllamaVisionModel (#9628)
Michael Goin committed about 2 years ago
[Bugfix][Model] Fix Mllama SDPA illegal memory access for batched multi-image (#9626)
Michael Goin committed about 2 years ago