A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 21:00 UTC
main
Luka Govedič committed over 1 year ago
[core] Bump ray to 2.43 (#13994)
Rui Qiao committed over 1 year ago
[v1] Move block pool operations to a separate class (#13973)
Chen Zhang committed over 1 year ago
Andrey Talman committed over 1 year ago
[v1] Cleanup the BlockTable in InputBatch (#13977)
Chen Zhang committed over 1 year ago
[V1][TPU] Integrate the new ragged paged attention kernel with vLLM v1 on TPU (#13379)
iefgnoix committed over 1 year ago
Fix entrypoint tests for embedding models (#14052)
Harry Mellor committed over 1 year ago
[Doc] Fix ROCm documentation (#14041)
Brayden Zhong committed over 1 year ago
[Bugfix] Initialize attention bias on the same device as Query/Key/Value for QwenVL Series (#14031)
Yang Liu committed over 1 year ago
[VLM][Bugfix] Enable specifying prompt target via index (#14038)
Cyrus Leung committed over 1 year ago
[Bugfix] Fix MoeWNA16Method activation (#14024)
Jee Jee Li committed over 1 year ago
Harry Mellor committed over 1 year ago
[V1][Minor] Restore V1 compatibility with LLMEngine class (#13090)
Thibault Schueller committed over 1 year ago
[Hardware][Intel-Gaudi] Regional compilation support (#13213)
Kacper Pietkun committed over 1 year ago
Use smaller embedding model when not testing model specifically (#13891)
Harry Mellor committed over 1 year ago
Mathis Felardos committed over 1 year ago
[Doc] Move multimodal Embedding API example to Online Serving page (#14017)
Cyrus Leung committed over 1 year ago
[Bugfix] Check that number of images matches number of <|image|> tokens with mllama (#13911)
Travis Johnson committed over 1 year ago
[V1]`SupportsV0Only` protocol for model definitions (#13959)
Roger Wang committed over 1 year ago
[Misc] Print FusedMoE detail info (#13974)
Jee Jee Li committed over 1 year ago