A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 19:45 UTC
main
[ROCm][FP8] Fix for adjustments needed only for fnuz (#14689)
Gregory Shtrasberg committed over 1 year ago
Add ray[data] as tpu dependency (#14691)
Richard Liu committed over 1 year ago
[bugfix] fixup warning message for plugged schedulers for v1 (#14700)
Joe Runde committed over 1 year ago
[release] Add force remove for TPU logs (#14697)
Kevin H. Luu committed over 1 year ago
[V1] Allow sliding window + prefix caching (#13069)
Woosuk Kwon committed over 1 year ago
[BugFix][V1] Fix parallel sampling finishing/aborts (#14512)
Nick Hill committed over 1 year ago
TJian committed over 1 year ago
[ROCm] Enable chunked prefill/paged attention in MLA on ROCm (#14316)
Sage Moore committed over 1 year ago
[Bugfix] Missing thumbnail from NVLM-D processor (#14633)
ameyanjarlekar committed over 1 year ago
[Model] Add support for Gemma 3 (#14660)
Woosuk Kwon committed over 1 year ago
[ROCm][Bugfix] Ensure that the moe_wna16_gemm kernel is not built on ROCm platforms. (#14629)
Sage Moore committed over 1 year ago
[CPU] Upgrade CPU backend to torch-2.6 (#13381)
Li, Jiang committed over 1 year ago
[Kernel] Add ModelOpt FP4 Checkpoint Support (#12520)
Pavani Majety committed over 1 year ago
Benjamin Chislett committed over 1 year ago
[Kernel] GGUF MoE kernel (#14613)
Szymon Ożóg committed over 1 year ago
[Core] Refactor `QKVCrossParallelLinear` implementation to support BNB 4-bit quantization (#14545)
Isotr0py committed over 1 year ago
[V1][Core] Support MistralTokenizer for Structured Output (#14625)
Aaron Pham committed over 1 year ago
[Model] Extend Ultravox to accept audio longer than 30s (#13631)
Farzad Abdolhosseini committed over 1 year ago
[Doc] Update benchmarks README (#14646)
Jennifer Zhao committed over 1 year ago
[core][V1] pluggable scheduler (#14466)
Joe Runde committed over 1 year ago