A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 22:00 UTC
main
[doc][misc] bump up py version in installation doc (#6119)
youkaichao committed over 2 years ago
[ Misc ] Clean Up `CompressedTensorsW8A8` (#6113)
Robert Shaw committed over 2 years ago
[vlm] Remove vision language config. (#6089)
xwjiang2010 committed over 2 years ago
[core][distributed] custom allreduce when pp size > 1 (#6117)
youkaichao committed over 2 years ago
[Kernel] Expand FP8 support to Ampere GPUs using FP8 Marlin (#5975)
Michael Goin committed over 2 years ago
[Bugfix] Fix `compute_logits` in Jamba (#6093)
Roger Wang committed over 2 years ago
Roger Wang committed over 2 years ago
[VLM][Frontend] Proper Image Prompt Formatting from OpenAI API (#6091)
Roger Wang committed over 2 years ago
[misc][distributed] error on invalid state (#6092)
youkaichao committed over 2 years ago
[BugFix] Avoid unnecessary Ray import warnings (#6079)
Nick Hill committed over 2 years ago
[CI] Fix base url doesn't strip "/" (#6087)
SangBin Cho committed over 2 years ago
[Core] Dynamic image size support for VLMs (#5276)
Cyrus Leung committed over 2 years ago
[hardware][misc] introduce platform abstraction (#6080)
youkaichao committed over 2 years ago
Mor Zusman committed over 2 years ago
[CORE] Quantized lm-head Framework (#4442)
Qubitium-ModelCloud committed over 2 years ago
[ Misc ] Refactor MoE to isolate Fp8 From Mixtral (#5970)
Robert Shaw committed over 2 years ago
Robert Shaw committed over 2 years ago
[Core] Pipeline Parallel Support (#4412)
Murali Andoorveedu committed over 2 years ago
[Speculative Decoding] MLPSpeculator Tensor Parallel support (1/2) (#6050)
Sirej Dua committed over 2 years ago
[Doc] Reinstate doc dependencies (#6061)
Cyrus Leung committed over 2 years ago