A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 00:30 UTC
main
[Minor] Fix the path typo in loader.py: save_sharded_states.py -> save_sharded_state.py (#5151)
Ye Cao committed over 2 years ago
[Kernel] Update Cutlass fp8 configs (#5144)
Varun Sundar Rabindranath committed over 2 years ago
[Kernel] Refactor CUTLASS kernels to always take scales that reside on the GPU (#5137)
Tyler Michael Smith committed over 2 years ago
[CI/Build] CMakeLists: build all extensions' cmake targets at the same time (#5034)
Daniele committed over 2 years ago
[Build] Guard against older CUDA versions when building CUTLASS 3.x kernels (#5168)
Tyler Michael Smith committed over 2 years ago
[Doc] Add checkmark for GPTBigCodeForCausalLM LoRA support (#5171)
Nick Hill committed over 2 years ago
[Model] Enable FP8 QKV in MoE and refine kernel tuning script (#5039)
Cody Yu committed over 2 years ago
[Misc]: optimize eager mode host time (#4196)
functionxu123 committed over 2 years ago
Simon Mo committed over 2 years ago
[Model] Support MAP-NEO model (#5081)
SnowDist committed over 2 years ago
Fix cutlass sm_90a vesrion in CMakeList
simon-mo committed over 2 years ago
Alexander Matveev committed over 2 years ago
[Bugfix] Avoid Warnings in SparseML Activation Quantization (#5120)
Robert Shaw committed over 2 years ago
[Build] Disable sm_90a in cu11 (#5141)
Simon Mo committed over 2 years ago
Bump version to v0.4.3 (#5046)
Simon Mo committed over 2 years ago
add doc about serving option on dstack (#3074)
Chansung Park committed over 2 years ago
[Doc] Use intersphinx and update entrypoints docs (#5125)
Cyrus Leung committed over 2 years ago
[Misc] remove duplicate definition of `seq_lens_tensor` in model_runner.py (#5129)
Hyunsung Lee committed over 2 years ago
[CI/Build] increase wheel size limit to 200 MB (#5130)
youkaichao committed over 2 years ago
[Bugfix] Automatically Detect SparseML models (#5119)
Robert Shaw committed over 2 years ago