A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 17:15 UTC
main
[moe][quant] add weight name case for offset (#15515)
Mengqing Cao committed over 1 year ago
[Doc] Update V1 user guide for fp8 kv cache support (#15585)
Wei Zeng committed over 1 year ago
[misc] LoRA: Remove unused long context test data (#15558)
Varun Sundar Rabindranath committed over 1 year ago
add platform check back (#15578)
Chenyaaang committed over 1 year ago
Add automatic tpu label to mergify.yml (#15560)
Michael Goin committed over 1 year ago
[Kernel] CUTLASS grouped gemm fp8 MoE kernel (#13972)
ElizaWszola committed over 1 year ago
Support FIPS enabled machines with MD5 hashing (#15299)
Matthew Vine committed over 1 year ago
[TPU] support disabling xla compilation cache (#15567)
Chengji Yao committed over 1 year ago
Use Cache Hinting for fused_moe kernel (#15511)
Wes committed over 1 year ago
[V1] TPU CI - Fix test_compilation.py (#15570)
Alexander Matveev committed over 1 year ago
[V1] TPU - Revert to exponential padding by default (#15565)
Alexander Matveev committed over 1 year ago
Applying some fixes for K8s agents in CI (#15493)
Alexei-V-Ivanov-AMD committed over 1 year ago
Support SHA256 as hash function in prefix caching (#15297)
marko committed over 1 year ago
[V1][Sampler] Faster top-k only implementation (#15478)
Nick Hill committed over 1 year ago
[Refactor] Remove unnecessary backend parameter in structured output interface (#15317)
Aaron Pham committed over 1 year ago
Fix weight loading for some models in Transformers backend (#15544)
Harry Mellor committed over 1 year ago
multi-node offline DP+EP example (#15484)
youkaichao committed over 1 year ago
[Model] Add Reasoning Parser for Granite Models (#14202)
Alex Brooks committed over 1 year ago
Improve validation of TP in Transformers backend (#15540)
Harry Mellor committed over 1 year ago
cyyever committed over 1 year ago