A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 04:30 UTC
main
[ci/build] Fix entrypoints test and pin outlines version (#11088)
Kevin H. Luu committed almost 2 years ago
[ci/build] Fix AMD CI dependencies (#11087)
Kevin H. Luu committed almost 2 years ago
[CI/Build] Check transformers v4.47 (#10991)
Cyrus Leung committed almost 2 years ago
Fix streaming for granite tool call when <|tool_call|> is present (#11069)
Maximilien de Bayser committed almost 2 years ago
[Model] PP support for Mamba-like models (#10992)
Mor Zusman committed almost 2 years ago
[Misc] LoRA + Chunked Prefill (#9057)
Aurick Qiao committed almost 2 years ago
[Bugfix] Fix Mamba multistep (#11071)
Tyler Michael Smith committed almost 2 years ago
[V1][Bugfix] Always set enable_chunked_prefill = True for V1 (#11061)
Woosuk Kwon committed almost 2 years ago
[torch.compile] add a flag to track batchsize statistics (#11059)
youkaichao committed almost 2 years ago
[Core] Update to outlines >= 0.1.8 (#10576)
Russell Bryant committed almost 2 years ago
[BUG] Remove token param #10921 (#11022)
Flávia Béo committed almost 2 years ago
[Bugfix] Backport request id validation to v0 (#11036)
Joe Runde committed almost 2 years ago
[Misc][LoRA] Add PEFTHelper for LoRA (#11003)
Jee Jee Li committed almost 2 years ago
[Bugfix] Handle <|tool_call|> token in granite tool parser (#11039)
Travis Johnson committed almost 2 years ago
Add example of helm chart for vllm deployment on k8s (#9199)
Maxime Fournioux committed almost 2 years ago
[Bugfix] cuda error running llama 3.2 (#11047)
Gene Der Su committed almost 2 years ago
Diego Marinho committed almost 2 years ago
[Bugfix] Fix xgrammar failing to read a vocab_size from LlavaConfig on PixtralHF. (#11043)
Jeff Cook committed almost 2 years ago
monitor metrics of tokens per step using cudagraph batchsizes (#11031)
youkaichao committed almost 2 years ago
[V1] Multiprocessing Tensor Parallel Support for v1 (#9856)
Tyler Michael Smith committed almost 2 years ago