A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 09:30 UTC
main
[Bugfix][Frontend] Guard against bad token ids (#9634)
Joe Runde committed about 2 years ago
[MISC] Set label value to timestamp over 0, to keep track of recent history (#9777)
Kunjan committed about 2 years ago
[CI/Build] mergify: fix rules for ci/build label (#9804)
Russell Bryant committed about 2 years ago
[CI][Bugfix] Skip chameleon for transformers 4.46.1 (#9808)
Michael Goin committed about 2 years ago
[Model] Add LlamaEmbeddingModel as an embedding Implementation of LlamaModel (#9806)
Junichi Sato committed about 2 years ago
[Misc]: Typo fix: Renaming classes (casualLM -> causalLM) (#9801)
yannicks1 committed about 2 years ago
[Bugfix] Use host argument to bind to interface (#9798)
Sven Seeberg committed about 2 years ago
Fix the log to correct guide user to install modelscope (#9793)
tastelikefeet committed about 2 years ago
[Misc] Add metrics for request queue time, forward time, and execute time (#9659)
科英 committed about 2 years ago
[Hardware] using current_platform.seed_everything (#9785)
wangshuai09 committed about 2 years ago
[Model] Add BNB quantization support for Mllama (#9720)
Isotr0py committed about 2 years ago
[Frontend] re-enable multi-modality input in the new beam search implementation (#9427)
Zhong Qishuai committed about 2 years ago
[Bugfix] Use temporary directory in registry (#9721)
Cyrus Leung committed about 2 years ago
[Doc] Specify async engine args in docs (#9726)
Cyrus Leung committed about 2 years ago
[Model][LoRA]LoRA support added for Qwen (#9622)
Jee Jee Li committed about 2 years ago
[Doc] fix third-party model example (#9771)
Russell Bryant committed about 2 years ago
[torch.compile] add deepseek v2 compile (#9775)
youkaichao committed about 2 years ago
[misc] avoid circular import (#9765)
youkaichao committed about 2 years ago
Adding "torch compile" annotations to moe models (#9758)
Yongzao committed about 2 years ago
[Model][VLM] Add multi-video support for LLaVA-Onevision (#8905)
litianjian committed about 2 years ago