A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 08:15 UTC
main
[CI/Build] Always run mypy (#10122)
Russell Bryant committed about 2 years ago
[Feature] [Spec decode]: Combine chunked prefill with speculative decoding (#9291)
Nicolò Lucchesi committed about 2 years ago
[Frontend] Tool calling parser for Granite 3.0 models (#9027)
Maximilien de Bayser committed about 2 years ago
[Misc] Add Gamma-Distribution Request Generation Support for Serving Benchmark. (#10105)
Atlas committed about 2 years ago
[Bugfix] Make image processor respect `mm_processor_kwargs` for Qwen2-VL (#10112)
Jiahao Li committed about 2 years ago
[Hardware][CPU][bugfix] Fix half dtype support on AVX2-only target (#10108)
Li, Jiang committed about 2 years ago
[Frontend] Fix multiple values for keyword argument error (#10075) (#10076)
Lei Yang committed about 2 years ago
Adds method to read the pooling types from model's files (#9506)
Flávia Béo committed about 2 years ago
[CI/Build] Improve mypy + python version matrix (#10041)
Russell Bryant committed about 2 years ago
[Core][Distributed] Refactor ipc buffer init in CustomAllreduce (#10030)
Hanzhi Zhou committed about 2 years ago
Doc: Improve benchmark documentation (#9927)
Rafael Vasquez committed about 2 years ago
[CI/Build] re-add codespell to CI (#10083)
Russell Bryant committed about 2 years ago
[Misc] Consolidate ModelConfig code related to HF config (#10104)
Cyrus Leung committed about 2 years ago
[V1][BugFix] Fix Generator construction in greedy + seed case (#10097)
Nick Hill committed about 2 years ago
[doc] add back Python 3.8 ABI (#10100)
youkaichao committed about 2 years ago
[Hardware][CPU] Update torch 2.5 (#9911)
Li, Jiang committed about 2 years ago
[Frontend] Adjust try/except blocks in API impl (#10056)
Nick Hill committed about 2 years ago
[Misc][XPU] Upgrade to Pytorch 2.5 for xpu backend (#9823)
Yan Ma committed about 2 years ago
[Bugfix] Fix FP8 torch._scaled_mm fallback for torch>2.5 with CUDA<12.4 (#10095)
Michael Goin committed about 2 years ago
[core][distributed] add stateless_init_process_group (#10072)
youkaichao committed about 2 years ago