A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 08:15 UTC
main
[Bugfix] Upgrade to pytorch 2.5.1 (#10001)
bnellnm committed about 2 years ago
[Frontend] Add max_tokens prometheus metric (#9881)
tomeras91 committed about 2 years ago
[Misc] Refactor benchmark_throughput.py (#9779)
lkchen committed about 2 years ago
[Bugfix] Fix `MQLLMEngine` hanging (#9973)
Robert Shaw committed about 2 years ago
[Doc] Update VLM doc about loading from local files (#9999)
Roger Wang committed about 2 years ago
[Bugfix][CI/Build][Hardware][AMD] Shard ID parameters in AMD tests running parallel jobs (#9279)
hissu-hyvarinen committed about 2 years ago
[CI] Basic Integration Test For TPU (#9968)
Robert Shaw committed about 2 years ago
[Model] factoring out MambaMixer out of Jamba (#8993)
Mor Zusman committed about 2 years ago
[Misc]Reduce BNB static variable (#9987)
Jee Jee Li committed about 2 years ago
[4/N] make quant config first-class citizen (#9978)
youkaichao committed about 2 years ago
[Frontend] Multi-Modality Support for Loading Local Image Files (#9915)
Chauncey committed about 2 years ago
[Bugfix][OpenVINO] Fix circular reference #9939 (#9974)
Mengqing Cao committed about 2 years ago
[Bugfix] Fix E2EL mean and median stats (#9984)
Tran Quang Dai committed about 2 years ago
[Misc] Compute query_start_loc/seq_start_loc on CPU (#9447)
Yang Zheng committed about 2 years ago
[Bugfix]Using the correct type hints (#9885)
Gregory Shtrasberg committed about 2 years ago
[Bugfix] Fix MiniCPMV and Mllama BNB bug (#9917)
Jee Jee Li committed about 2 years ago
Robert Shaw committed about 2 years ago
[Model] Add support for H2OVL-Mississippi models (#9747)
shanshan wang committed about 2 years ago
[V1] Support per-request seed (#9945)
Nick Hill committed about 2 years ago
youkaichao committed about 2 years ago