A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 00:45 UTC
main
Squelch MLA warning for Compressed-Tensors Models (#12704)
Kyle Sayers committed almost 2 years ago
[Bugfix][Kernel] Fix per-token/per-channel quantization for Hopper scaled mm (#12696)
Tyler Michael Smith committed almost 2 years ago
[Misc] Fix improper placement of SPDX header in scripts (#12694)
Russell Bryant committed almost 2 years ago
[Model]: Add `transformers` backend support (#11330)
Arthur committed almost 2 years ago
[ci/build] fix gh200 test (#12681)
youkaichao committed almost 2 years ago
[cuda] manually import the correct pynvml module (#12679)
youkaichao committed almost 2 years ago
Fix for attention layers to remain unquantized during moe_wn16 quant (#12570)
Srikanth Srinivas committed almost 2 years ago
Properly check if all fused layers are in the list of targets (#12666)
Eldar Kurtic committed almost 2 years ago
make sure mistral_common not imported for non-mistral models (#12669)
youkaichao committed almost 2 years ago
[Kernel] port sgl moe_align_block_size kernels (#12574)
Yang Chen committed almost 2 years ago
[Doc] Deprecate Discord (#12668)
Zhuohan Li committed almost 2 years ago
[doc][misc] clarify VLLM_HOST_IP for multi-node inference (#12667)
youkaichao committed almost 2 years ago
[Misc] Add SPDX-License-Identifier headers to python source files (#12628)
Russell Bryant committed almost 2 years ago
[Hardware][Intel GPU] add XPU bf16 support (#12392)
Kunshang Ji committed almost 2 years ago
[Core][v1] Unify allocating slots in prefill and decode in KV cache manager (#12608)
Shawn Du committed almost 2 years ago
[V1][Minor] Avoid frequently creating ConstantList (#12653)
Woosuk Kwon committed almost 2 years ago
[Core] Silence unnecessary deprecation warnings (#12620)
Russell Bryant committed almost 2 years ago
[Bugfix] fix moe_wna16 get_quant_method (#12648)
Jinzhen Lin committed almost 2 years ago
doc: fixing minor typo in readme.md (#12643)
Vicente Herrera committed almost 2 years ago
Apply torch.compile to fused_moe/grouped_topk (#12637)
Michael Goin committed almost 2 years ago