A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 22:00 UTC
main
[Doc] Reorganize Supported Models by Type (#6167)
Roger Wang committed over 2 years ago
Move release wheel env var to Dockerfile instead (#6163)
Simon Mo committed over 2 years ago
Fix release wheel build env var (#6162)
Simon Mo committed over 2 years ago
Update wheel builds to strip debug (#6161)
Simon Mo committed over 2 years ago
[Docs] Fix readthedocs for tag build (#6158)
Simon Mo committed over 2 years ago
bump version to v0.5.1 (#6157)
Simon Mo committed over 2 years ago
[Bugfix] Add verbose error if scipy is missing for blocksparse attention (#5695)
JGSweets committed over 2 years ago
[Frontend] Continuous usage stats in OpenAI completion API (#5742)
jvlunteren committed over 2 years ago
[Bugfix] Use templated datasource in grafana.json to allow automatic imports (#6136)
Christian Rohmann committed over 2 years ago
[VLM] Cleanup validation and update docs (#6149)
Cyrus Leung committed over 2 years ago
[VLM] Improve consistency between feature size calculation and dummy data for profiling (#6146)
Roger Wang committed over 2 years ago
[VLM] Calculate maximum number of multi-modal tokens by model (#6121)
Cyrus Leung committed over 2 years ago
[Kernel][Model] logits_soft_cap for Gemma2 with flashinfer (#6051)
Lily Liu committed over 2 years ago
[Hardware][Intel CPU] Adding intel openmp tunings in Docker file (#6008)
Yuan committed over 2 years ago
[misc][doc] try to add warning for latest html (#5979)
youkaichao committed over 2 years ago
Gregory Shtrasberg committed over 2 years ago
[CI/Build] Cleanup VLM tests (#6107)
Cyrus Leung committed over 2 years ago
[Distributed][Core] Support Py39 and Py38 for PP (#6120)
Murali Andoorveedu committed over 2 years ago
[Bugfix] set OMP_NUM_THREADS to 1 by default for multiprocessing (#6109)
Travis Johnson committed over 2 years ago
[core][distributed] support n layers % pp size != 0 (#6115)
youkaichao committed over 2 years ago