A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 10:45 UTC
main
[Bugfix] Block manager v2 with preemption and lookahead slots (#8824)
sroy745 committed about 2 years ago
[Core] Improve choice of Python multiprocessing method (#8823)
Russell Bryant committed about 2 years ago
[doc] organize installation doc and expose per-commit docker (#8931)
youkaichao committed about 2 years ago
[Bugfix] Support testing prefill throughput with benchmark_serving.py --hf-output-len 1 (#8891)
Chen Zhang committed about 2 years ago
[Frontend] Make beam search emulator temperature modifiable (#8928)
Edouard B. committed about 2 years ago
[CI/Build] Update models tests & examples (#8874)
Cyrus Leung committed about 2 years ago
[Bugfix] Fix PP for Multi-Step (#8887)
Varun Sundar Rabindranath committed about 2 years ago
[Bugfix] Fix code for downloading models from modelscope (#8443)
tastelikefeet committed about 2 years ago
[Misc] Remove vLLM patch of `BaichuanTokenizer` (#8921)
Cyrus Leung committed about 2 years ago
[Bugfix][Intel] Fix XPU Dockerfile Build (#7824)
Tyler Titsworth committed about 2 years ago
youkaichao committed about 2 years ago
[Core] Priority-based scheduling in async engine (#8850)
Sebastian Schoennenbeck committed about 2 years ago
[misc][distributed] add VLLM_SKIP_P2P_CHECK flag (#8911)
youkaichao committed about 2 years ago
[Core] Multi-Step + Single Step Prefills via Chunked Prefill code path (#8378)
Varun Sundar Rabindranath committed about 2 years ago
[Bugfix] fix for deepseek w4a16 (#8906)
Lucas Wilkinson committed about 2 years ago
[Kernel] AQ AZP 4/4: Integrate asymmetric quantization to linear method (#7271)
Luka Govedič committed about 2 years ago
[torch.compile] use empty tensor instead of None for profiling (#8875)
youkaichao committed about 2 years ago
[TPU] Update pallas.py to support trillium (#8871)
Brittany committed about 2 years ago
[Bugfix][VLM] Fix Fuyu batching inference with `max_num_seqs>1` (#8892)
Isotr0py committed about 2 years ago
[MISC] Fix invalid escape sequence '\' (#8830)
Peter Pan committed about 2 years ago