A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 10:45 UTC
main
Add classifiers in setup.py (#9171)
Yuan Tang committed about 2 years ago
support bitsandbytes quantization with more models (#9148)
chenqianfzh committed about 2 years ago
Add `lm-eval` directly to requirements-test.txt (#9161)
Michael Goin committed about 2 years ago
mypy: check additional directories (#9162)
Russell Bryant committed about 2 years ago
[Bugfix] Try to handle older versions of pytorch (#9086)
bnellnm committed about 2 years ago
[Doc] Improve contributing and installation documentation (#9132)
Rafael Vasquez committed about 2 years ago
[Doc] Update vlm.rst to include an example on videos (#9155)
Sayak Paul committed about 2 years ago
[Bugfix] fix OpenAI API server startup with --disable-frontend-multiprocessing (#8537)
Daniele committed about 2 years ago
Peter Pan committed about 2 years ago
[Frontend] Add Early Validation For Chat Template / Tool Call Parser (#9151)
Alex Brooks committed about 2 years ago
[Core][Frontend] Add Support for Inference Time mm_processor_kwargs (#9131)
Alex Brooks committed about 2 years ago
[Frontend] API support for beam search for MQLLMEngine (#9117)
Brendan Wong committed about 2 years ago
[misc] improve ux on readme (#9147)
youkaichao committed about 2 years ago
[Intel GPU] Fix xpu decode input (#9145)
Kunshang Ji committed about 2 years ago
[misc] update utils to support comparing multiple settings (#9140)
youkaichao committed about 2 years ago
Simon Mo committed about 2 years ago
[misc] fix comment and variable name (#9139)
youkaichao committed about 2 years ago
[Doc] Include performance benchmark in README (#9135)
Kuntai Du committed about 2 years ago
[CI/Build] Add linting for github actions workflows (#7876)
Russell Bryant committed about 2 years ago
[Doc]: Add deploying_with_k8s guide (#8451)
TimWang committed about 2 years ago