A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 00:30 UTC
main
[Model] Add support for falcon-11B (#5069)
Isotr0py committed over 2 years ago
[Core] Allow AQLM on Pascal (#5058)
sasha0552 committed over 2 years ago
[Bugfix / Core] Prefix Caching Guards (merged with main) (#4846)
Zhuohan Li committed over 2 years ago
[Misc] Make Serving Benchmark More User-friendly (#5044)
Roger Wang committed over 2 years ago
[Dynamic Spec Decoding] Minor fix for disabling speculative decoding (#5000)
Lily Liu committed over 2 years ago
[Misc] add logging level env var (#5045)
youkaichao committed over 2 years ago
[Kernel][Backend][Model] Blocksparse flash attention kernel and Phi-3-Small model (#4799)
Eric Xihui Lin committed over 2 years ago
[Core][Bugfix]: fix prefix caching for blockv2 (#4764)
leiwen83 committed over 2 years ago
[Bugfix] Fix Mistral v0.3 Weight Loading (#5005)
Robert Shaw committed over 2 years ago
[Doc] add ccache guide in doc (#5012)
youkaichao committed over 2 years ago
[Core]: Option To Use Prompt Token Ids Inside Logits Processor (#4985)
Elisei Smirnov committed over 2 years ago
[Kernel] Initial Activation Quantization Support (#4525)
Dipika Sikka committed over 2 years ago
[Core][1/N] Support send/recv in PyNCCL Groups (#4988)
Murali Andoorveedu committed over 2 years ago
[Bugfix] Update Dockerfile.cpu to fix NameError: name 'vllm_ops' is not defined (#5009)
Letian Li committed over 2 years ago
Marlin 24 prefill performance improvement (about 25% better on average) (#4983)
Alexander Matveev committed over 2 years ago
[Misc] Take user preference in attention selector (#4960)
Cody Yu committed over 2 years ago
[Minor] Fix small typo in llama.py: QKVParallelLinear -> QuantizationConfig (#4991)
Philipp Moritz committed over 2 years ago
[Core] Eliminate parallel worker per-step task scheduling overhead (#4894)
Nick Hill committed over 2 years ago
[Model] LoRA gptbigcode implementation (#3949)
raywanb committed over 2 years ago
[Misc] Load FP8 kv-cache scaling factors from checkpoints (#4893)
Cody Yu committed over 2 years ago