A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 19:45 UTC
main
[Misc] Add log information for handle_process_request. (#14130)
Chauncey committed over 1 year ago
[Feature] Consolidate performance benchmark datasets (#14036)
Jennifer Zhao committed over 1 year ago
[Bugfix] Wrong requirements path - rocm (#14527)
Martin Hoyer committed over 1 year ago
[Misc] Ensure out-of-tree quantization method recognize by cli args (#14328)
Yanyi Liu committed over 1 year ago
[Hardware][TPU] Fix the recompiling issue in logits processor after warmup (#14510)
Chengji Yao committed over 1 year ago
[Bugfix] Revert QKVCrossParallelLinear usage in Mllama to keep BNB quantization work (#14498)
Isotr0py committed over 1 year ago
[Bugfix] Fix tqdm progress bar when SamplingParams.n > 1 (#12428)
Yuchen Yan committed over 1 year ago
[Feat] Support chunked prefill for LMCache connector (#14505)
Jiayi Yao committed over 1 year ago
[V1][TPU] Remove unnecessary padding for running on TPU. (#14467)
iefgnoix committed over 1 year ago
[Attention] Default to FlashMLA backend for MLA (#14451)
Lucas Wilkinson committed over 1 year ago
Revert "[V1][Core] Fix memory issue with logits & sampling" (#14504)
Robert Shaw committed over 1 year ago
[V1] Support bad_words in sampler (#13376)
22quinn committed over 1 year ago
[Misc] Upgrade to Python 3.9 typing for additional directories (#14492)
Cyrus Leung committed over 1 year ago
Update CODEOWNERS for structured output (#14496)
Russell Bryant committed over 1 year ago
[Bugfix] Fix profiling OOM and decouple encoder multimodal profiling (#14361)
Isotr0py committed over 1 year ago
[Bugfix] DeepSeek Accuracy (#14476)
Lucas Wilkinson committed over 1 year ago
Move requirements into their own directory (#12547)
Harry Mellor committed over 1 year ago
[Misc] Don't run ruff at all on 3rd party libs (#14493)
Cyrus Leung committed over 1 year ago
[benchmarks] Add option to use unique jsonschema for each request (#14457)
Russell Bryant committed over 1 year ago
[V1][Core] Fix memory issue with logits & sampling (#13776)
Roger Wang committed over 1 year ago