A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 10:45 UTC
main
[Performance][Spec Decode] Optimize ngram lookup performance (#9333)
Lily Liu committed about 2 years ago
[Model][Bugfix] Add FATReLU activation and support for openbmb/MiniCPM-S-1B-sft (#9396)
Junhao Li committed about 2 years ago
[Kernel][Model] Improve continuous batching for Jamba and Mamba (#9189)
Mor Zusman committed about 2 years ago
Support mistral interleaved attn (#9414)
Patrick von Platen committed about 2 years ago
[Model] Support SDPA attention for Molmo vision backbone (#9410)
Isotr0py committed about 2 years ago
[Misc] Consolidate example usage of OpenAI client for multimodal models (#9412)
Roger Wang committed about 2 years ago
[Core] Rename input data types (#8688)
Cyrus Leung committed about 2 years ago
[CI/Build] Test VLM embeddings (#9406)
Cyrus Leung committed about 2 years ago
[Model] VLM2Vec, the first multimodal embedding model in vLLM (#9303)
Cyrus Leung committed about 2 years ago
[Misc] Standardize RoPE handling for Qwen2-VL (#9250)
Cyrus Leung committed about 2 years ago
[Bugfix] Molmo text-only input bug fix (#9397)
Reza Salehi committed about 2 years ago
[Bugfix][CI/Build] Fix CUDA 11.8 Build (#9386)
Lucas Wilkinson committed about 2 years ago
[Bugfix] Fix vLLM UsageInfo and logprobs None AssertionError with empty token_ids (#9034)
Chang Su committed about 2 years ago
[Misc] Directly use compressed-tensors for checkpoint definitions (#8909)
Michael Goin committed about 2 years ago
pass ignore_eos parameter to all benchmark_serving calls (#9349)
Grace Ho committed about 2 years ago
[BugFix] Fix chat API continuous usage stats (#9357)
Nick Hill committed about 2 years ago
[Bugfix] Update InternVL input mapper to support image embeds (#9351)
hhzhang16 committed about 2 years ago
[Doc] Fix code formatting in spec_decode.rst (#9348)
Michael Goin committed about 2 years ago
[Frontend] Clarify model_type error messages (#9345)
Steve Grubb committed about 2 years ago
[Bugfix] Clean up some cruft in mamba.py (#9343)
Tyler Michael Smith committed about 2 years ago