A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 09:30 UTC
main
[CI/Build] Simplify exception trace in api server tests (#9787)
Yongzao committed about 2 years ago
[doc] update pp support (#9853)
youkaichao committed about 2 years ago
[Neuron] Update Dockerfile.neuron to fix build failure (#9822)
Harsha vardhan manoj Bikki committed about 2 years ago
[Doc] link bug for multistep guided decoding (#9843)
Joe Runde committed about 2 years ago
[Bugfix][core] replace heartbeat with pid check (#9818)
Joe Runde committed about 2 years ago
[Misc] Add chunked-prefill support on FlashInfer. (#9781)
Elfie Guo committed about 2 years ago
[Model] Support math-shepherd-mistral-7b-prm model (#9697)
Went-Liang committed about 2 years ago
[CI/Build] VLM Test Consolidation (#9372)
Alex Brooks committed about 2 years ago
[TPU] Correctly profile peak memory usage & Upgrade PyTorch XLA (#9438)
Woosuk Kwon committed about 2 years ago
[Misc] Specify minimum pynvml version (#9827)
Jee Jee Li committed about 2 years ago
[torch.compile] rework compile control with piecewise cudagraph (#9715)
youkaichao committed about 2 years ago
[Doc] Add the DCO to CONTRIBUTING.md (#9803)
Russell Bryant committed about 2 years ago
[Bugfix] Fix multi nodes TP+PP for XPU (#8884)
Yan Ma committed about 2 years ago
[ci/build] Pin CI dependencies version with pip-compile (#9810)
Kevin H. Luu committed about 2 years ago
[Bugfix][VLM] Make apply_fp8_linear work with >2D input (#9812)
Michael Goin committed about 2 years ago
Lily Liu committed about 2 years ago
[core][distributed] fix custom allreduce in pytorch 2.5 (#9815)
youkaichao committed about 2 years ago
[Bugfix] Fix prefix strings for quantized VLMs (#9772)
Michael Goin committed about 2 years ago
[Docs] Add notes about Snowflake Meetup (#9814)
Simon Mo committed about 2 years ago
[Model] tool calling support for ibm-granite/granite-20b-functioncalling (#8339)
Will Eaton committed about 2 years ago