A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 07:00 UTC
main
[Bugfix] Fix tensor parallel for qwen2 classification model (#10297)
Isotr0py committed about 2 years ago
[Bugfix] bitsandbytes models fail to run pipeline parallel (#10200)
HoangCongDuc committed about 2 years ago
[1/N] Initial prototype for multi-modal processor (#10044)
Cyrus Leung committed about 2 years ago
[V1] Add missing tokenizer options for `Detokenizer` (#10288)
Roger Wang committed about 2 years ago
[Model][LoRA]LoRA support added for idefics3 (#10281)
B-201 committed about 2 years ago
[Core] Flashinfer - Remove advance step size restriction (#10282)
Pavani Majety committed about 2 years ago
[Model] Adding Support for Qwen2VL as an Embedding Model. Using MrLight/dse-qwen2-2b-mrl-v1 (#9944)
Austin Veselka committed about 2 years ago
电脑星人 committed about 2 years ago
[Doc] Fix typo in arg_utils.py (#10264)
Xin Yang committed about 2 years ago
Bump to compressed-tensors v0.8.0 (#10279)
Dipika Sikka committed about 2 years ago
[V1] Support VLMs with fine-grained scheduling (#9871)
Woosuk Kwon committed about 2 years ago
[core][distributed] use tcp store directly (#10275)
youkaichao committed about 2 years ago
[V1] Fix CI tests on V1 engine (#10272)
Woosuk Kwon committed about 2 years ago
Revert "[ci][build] limit cmake version" (#10271)
youkaichao committed about 2 years ago
[doc] improve debugging doc (#10270)
youkaichao committed about 2 years ago
[doc] fix location of runllm widget (#10266)
youkaichao committed about 2 years ago
[V1] Enable Inductor when using piecewise CUDA graphs (#10268)
Woosuk Kwon committed about 2 years ago
[LoRA] Adds support for bias in LoRA (#5733)
Umesh committed about 2 years ago
[Encoder Decoder] Update Mllama to run with both FlashAttention and XFormers (#9982)
sroy745 committed about 2 years ago
Woosuk Kwon committed about 2 years ago