A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 23:30 UTC
main
[misc] Add LoRA to benchmark_serving (#12898)
Varun Sundar Rabindranath committed almost 2 years ago
[Hardware][Intel-Gaudi] Enable long-contexts + LoRA support for Intel Gaudi (#12812)
Sanju C Sudhakaran committed almost 2 years ago
[Build] Make pypi install work on CPU platform (#12874)
wangxiyuan committed almost 2 years ago
[V1][Minor] Move cascade attn logic outside _prepare_inputs (#12943)
Woosuk Kwon committed almost 2 years ago
[Misc] Add offline test for disaggregated prefill (#12418)
Shaoting committed almost 2 years ago
[bugfix] respect distributed_executor_backend in world_size=1 (#12934)
youkaichao committed almost 2 years ago
[Bugfix] Fix multi-round chat error when mistral tokenizer is used (#12859)
zifeitong committed almost 2 years ago
[Misc] Fix typo in the example file (#12896)
Ke Zhao committed almost 2 years ago
[Bugfix] Fix Qwen2_5_VLForConditionalGeneration packed_modules_mapping (#12905)
Jee Jee Li committed almost 2 years ago
[V1] Move KV block hashes from Request to KVCacheManager (#12922)
Woosuk Kwon committed almost 2 years ago
[V1][Minor] Remove outdated comment (#12928)
Woosuk Kwon committed almost 2 years ago
[Bugfix] Fix disagg hang caused by the prefill and decode communication issues (#12723)
Lu Fang committed almost 2 years ago
[V1] LM Eval With Streaming Integration Tests (#11590)
Robert Shaw committed almost 2 years ago
TJian committed almost 2 years ago
[V1] Logprobs and prompt logprobs support (#9880)
afeldman-nm committed almost 2 years ago
Amit Garg committed almost 2 years ago
[Misc] Remove unnecessary detokenization in multimodal processing (#12868)
Cyrus Leung committed almost 2 years ago
[MISC][EASY] Break check file names into entry and args in the pre-commit hooks (#12880)
Lu Fang committed almost 2 years ago
Prevent unecessary requests to huggingface hub (#12837)
Maximilien de Bayser committed almost 2 years ago
[Bugfix] Missing quant_config in deepseek embedding layer (#12836)
Szymon Ożóg committed almost 2 years ago