A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 02:00 UTC
main
[core] Turn off GPU communication overlap for Ray executor (#12051)
Rui Qiao committed almost 2 years ago
[Misc] Add multipstep chunked-prefill support for FlashInfer (#10467)
Elfie Guo committed almost 2 years ago
[V1][BugFix] Fix edge case in VLM scheduling (#12065)
Woosuk Kwon committed almost 2 years ago
[Platform] move current_memory_usage() into platform (#11369)
Shanshan Shen committed almost 2 years ago
[HPU][Bugfix] Don't use /dev/accel/accel0 for HPU autodetection in setup.py (#12046)
Konrad Zawora committed almost 2 years ago
[Kernel] Support MulAndSilu (#11624)
Jee Jee Li committed almost 2 years ago
[Misc] Merge bitsandbytes_stacked_params_mapping and packed_modules_mapping (#11924)
Jee Jee Li committed almost 2 years ago
[Doc]: Update the Json Example of the `Engine Arguments` document (#12045)
maang-h committed almost 2 years ago
Explain where the engine args go when using Docker (#12041)
Harry Mellor committed almost 2 years ago
[Bugfix][Kernel] Give unique name to BlockSparseFlashAttention (#12040)
Chen Zhang committed almost 2 years ago
[Platform] Add output for Attention Backend (#11981)
wangxiyuan committed almost 2 years ago
[Kernel] Revert the API change of Attention.forward (#12038)
Chen Zhang committed almost 2 years ago
[Bugfix] Fix various bugs in multi-modal processor (#12031)
Cyrus Leung committed almost 2 years ago
[HPU][misc] add comments for explanation (#12034)
youkaichao committed almost 2 years ago
[Doc] Update Quantization Hardware Support Documentation (#12025)
TJian committed almost 2 years ago
[HPU][Bugfix] set_forward_context and CI test execution (#12014)
Konrad Zawora committed almost 2 years ago
[Docs] Add Sky Computing Lab to project intro (#12019)
Woosuk Kwon committed almost 2 years ago
[Bugfix] Fix deepseekv3 gate bias error (#12002)
Steve Luo committed almost 2 years ago
[Doc] Fix build from source and installation link in README.md (#12013)
Yikun Jiang committed almost 2 years ago
bugfix: Fix signature mismatch in benchmark's `get_tokenizer` function (#11982)
elijah committed almost 2 years ago