A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 00:45 UTC
main
[Bugfix] Gracefully handle huggingface hub http error (#12571)
Roger Wang committed almost 2 years ago
[Attention] MLA decode optimizations (#12528)
Lucas Wilkinson committed almost 2 years ago
[ROCm][AMD][Model] llama 3.2 support upstreaming (#12421)
Aleksandr Malyshev committed almost 2 years ago
[Kernel] Update `cutlass_scaled_mm` to support 2d group (blockwise) scaling (#11868)
Lucas Wilkinson committed almost 2 years ago
[V1][Log] Add max request concurrency log to V1 (#12569)
Michael Goin committed almost 2 years ago
[CPU][PPC] Updated torch, torchvision, torchaudio dependencies (#12555)
Nishidha committed almost 2 years ago
[Kernel] Triton Configs for Fp8 Block Quantization (#11589)
Robert Shaw committed almost 2 years ago
[Misc] fix typo: add missing space in lora adapter error message (#12564)
Beim committed almost 2 years ago
Set `?device={device}` when changing tab in installation guides (#12560)
Harry Mellor committed almost 2 years ago
[V1][Metrics] Add GPU cache usage % gauge (#12561)
Mark McLoughlin committed almost 2 years ago
[Misc][MoE] add Deepseek-V3 moe tuning support (#12558)
Divakar Verma committed almost 2 years ago
[V1][BugFix] Free encoder cache for aborted requests (#12545)
Woosuk Kwon committed almost 2 years ago
Revert "[Build/CI] Fix libcuda.so linkage" (#12552)
Tyler Michael Smith committed almost 2 years ago
[Kernel] add triton fused moe kernel for gptq/awq (#12185)
Jinzhen Lin committed almost 2 years ago
[Hardware][NV] Fix Modelopt model loading for k-v-scales for Llama models. (#11787)
Pavani Majety committed almost 2 years ago
[Frontend] Support override generation config in args (#12409)
Yanyi Liu committed almost 2 years ago
[Model] Refactoring of MiniCPM-V and add MiniCPM-o-2.6 support for vLLM (#12069)
Alphi committed almost 2 years ago
[Bugfix] handle alignment of arguments in convert_sparse_cross_attention_mask_to_dense (#12347)
Travis Johnson committed almost 2 years ago
Fix the pydantic logging validator (#12420)
Maximilien de Bayser committed almost 2 years ago
[V1] Improve Error Message for Unsupported Config (#12535)
Robert Shaw committed almost 2 years ago