A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 03:00 UTC
main
[Misc] Fix typo in scheduler.py (#4022)
Zhuohan Li committed over 2 years ago
[Bugfix] fix type hint for py 3.8 (#4036)
youkaichao committed over 2 years ago
[Test] Test multiple attn backend for chunked prefill. (#4023)
SangBin Cho committed over 2 years ago
[Frontend][Core] Move `merge_async_iterators` to utils (#4026)
Cyrus Leung committed over 2 years ago
[Core] Support LoRA on quantized models (#4012)
Jee Li committed over 2 years ago
[Doc] Add typing hints / mypy types cleanup (#3816)
Michael Feil committed over 2 years ago
[BugFix] Fix handling of stop strings and stop token ids (#3672)
Nick Hill committed over 2 years ago
Add extra punica sizes to support bigger vocabs (#4015)
Antoni Baum committed over 2 years ago
Fix echo/logprob OpenAI completion bug (#3441)
Dylan Hawk committed over 2 years ago
[Core] init_distributed_environment align with init_process_group(#4014)
youkaichao committed over 2 years ago
[Core] Set `linear_weights` directly on the layer (#3977)
Antoni Baum committed over 2 years ago
[Hardware][Intel] Isolate CPUModelRunner and ModelRunner for better maintenance (#3824)
bigPYJ1151 committed over 2 years ago
punica fix-bgmv-kernel-640 (#4007)
fuchen.ljl committed over 2 years ago
[Kernel] Fused MoE Config for Mixtral 8x22 (#4002)
Roger Wang committed over 2 years ago
[Doc][Installation] delete python setup.py develop (#3989)
youkaichao committed over 2 years ago
[Misc] Add indirection layer for custom ops (#3913)
Kunshang Ji committed over 2 years ago
[Test] Add xformer and flash attn tests (#3961)
SangBin Cho committed over 2 years ago
[Core][Model] torch.compile for layernorm in commandr (#3985)
youkaichao committed over 2 years ago
[Core][5/N] Fully working chunked prefill e2e (#3884)
SangBin Cho committed over 2 years ago
[Core][Refactor] move parallel_utils into vllm/distributed (#3950)
youkaichao committed over 2 years ago