A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 10:45 UTC
main
[Kernel] adding fused moe kernel config for L40S TP4 (#9245)
Burkhard Ringlein committed about 2 years ago
Tyler Michael Smith committed about 2 years ago
[Bugfix] Fix priority in multiprocessing engine (#9277)
Sebastian Schoennenbeck committed about 2 years ago
[Misc][LoRA] Support loading LoRA weights for target_modules in reg format (#9275)
Jee Jee Li committed about 2 years ago
[Misc] Collect model support info in a single process per model (#9233)
Cyrus Leung committed about 2 years ago
[misc] hide best_of from engine (#9261)
youkaichao committed about 2 years ago
[Misc] Fix sampling from sonnet for long context case (#9235)
Andy Dai committed about 2 years ago
[Doc][Neuron] add note to neuron documentation about resolving triton issue (#9257)
omrishiv committed about 2 years ago
[torch.compile] generic decorators (#9258)
youkaichao committed about 2 years ago
[ci/build] Add placeholder command for custom models test (#9262)
Kevin H. Luu committed about 2 years ago
Bump actions/setup-python from 3 to 5 (#9195)
dependabot[bot] committed about 2 years ago
Bump actions/checkout from 3 to 4 (#9196)
dependabot[bot] committed about 2 years ago
Bump actions/github-script from 6 to 7 (#9197)
dependabot[bot] committed about 2 years ago
[torch.compile] integration with compilation control (#9058)
youkaichao committed about 2 years ago
Suggest codeowners for the core componenets (#9210)
Simon Mo committed about 2 years ago
jordanyono committed about 2 years ago
[Doc] Improve debugging documentation (#9204)
Rafael Vasquez committed about 2 years ago
[Bugfix] Fix Machete unittests failing with `NotImplementedError` (#9218)
Lucas Wilkinson committed about 2 years ago
[OpenVINO] Use torch 2.4.0 and newer optimim version (#9121)
Ilya Lavrenov committed about 2 years ago
[Model] support input image embedding for minicpmv (#9237)
whyiug committed about 2 years ago