A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 03:00 UTC
main
[Bugfix] Fix ray workers profiling with nsight (#4095)
Ricky Xu committed over 2 years ago
[Doc] Add better clarity for tensorizer usage (#4090)
Sanger Steel committed over 2 years ago
[Core] Simplifications to executor classes (#4071)
Nick Hill committed over 2 years ago
[Misc][Minor] Fix CPU block num log in CPUExecutor. (#4088)
Li, Jiang committed over 2 years ago
[Bugfix] Fix filelock version requirement (#4075)
Zhuohan Li committed over 2 years ago
[Misc] Upgrade triton to 2.2.0 (#4061)
Roy committed over 2 years ago
[Docs] document that mixtral 8x22b is supported (#4073)
Simon Mo committed over 2 years ago
[BugFix] Fix tensorizer extra in setup.py (#4072)
Nick Hill committed over 2 years ago
[Core] avoid too many cuda context by caching p2p test (#4021)
youkaichao committed over 2 years ago
[Frontend] [Core] feat: Add model loading using `tensorizer` (#3476)
Sanger Steel committed over 2 years ago
[Kernel] Add punica dimension for Baichuan-13B (#4053)
Jee Li committed over 2 years ago
[Bugfix] fix_small_bug_in_neuron_executor (#4051)
zspo committed over 2 years ago
[Bugfix] fix_log_time_in_metrics (#4050)
zspo committed over 2 years ago
[Core][Distributed] improve logging for init dist (#4042)
youkaichao committed over 2 years ago
[Bugfix] More type hint fixes for py 3.8 (#4039)
Dylan Hawk committed over 2 years ago
[CI/Test] expand ruff and yapf for all supported python version (#4037)
youkaichao committed over 2 years ago
Jee Li committed over 2 years ago
Fix triton compilation issue (#3984)
Bellk17 committed over 2 years ago
[Core] fix custom allreduce default value (#4040)
youkaichao committed over 2 years ago
[mypy] Add mypy type annotation part 1 (#4006)
SangBin Cho committed over 2 years ago