A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 16:00 UTC
main
[Spec Decode] Fix input triton kernel for eagle (#15909)
Ekagra Ranjan committed over 1 year ago
[Docs] Add Intel as Sponsor (#15913)
Simon Mo committed over 1 year ago
[Docs] Add Ollama meetup slides (#15905)
Simon Mo committed over 1 year ago
[Model][MiniMaxText01] Support MiniMaxText01 model inference (#13454)
Gerald committed over 1 year ago
[BugFix] make sure socket close (#15875)
yihong committed over 1 year ago
[Docs] update usage stats language (#15898)
Simon Mo committed over 1 year ago
[V1][Spec Decode] Implement Eagle Proposer [1/N] (#15729)
Woosuk Kwon committed over 1 year ago
[V1][Metrics] Initial speculative decoding metrics (#15151)
Mark McLoughlin committed over 1 year ago
[CI] Disable flaky structure decoding test temporarily. (#15892)
Roger Wang committed over 1 year ago
[Model] Add module name prefixes to gemma3 (#15889)
cloud11665 committed over 1 year ago
Jennifer Zhao committed over 1 year ago
[CI/Build] Clean up LoRA tests (#15867)
Jee Jee Li committed over 1 year ago
Add option to use DeepGemm contiguous grouped gemm kernel for fused MoE operations. (#13932)
bnellnm committed over 1 year ago
[ROCm][Build][Bugfix] Bring the base dockerfile in sync with the ROCm fork (#15820)
Gregory Shtrasberg committed over 1 year ago
[Misc] Allow using OpenCV as video IO fallback (#15055)
Isotr0py committed over 1 year ago
Reinstate `format.sh` and make `pre-commit` installation simpler (#15890)
Harry Mellor committed over 1 year ago
[Doc] Quark quantization documentation (#15861)
chaow-amd committed over 1 year ago
[New Model]: jinaai/jina-reranker-v2-base-multilingual (#15876)
wang.yuqi committed over 1 year ago
Remove `format.sh` as it's been unsupported >70 days (#15884)
Harry Mellor committed over 1 year ago
[Misc] remove unused script (#15746)
Reid committed over 1 year ago