A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 19:45 UTC
main
[BugFix]: properly catch templating error when preprocess input (#13976)
Guillaume Calmettes committed over 1 year ago
[VLM] Various cleanup and fixes (#14806)
Cyrus Leung committed over 1 year ago
[Bugfix][W8A8] fixed cutlass block fp8 binding (#14796)
DefTruth committed over 1 year ago
[V1][Minor] Minor code cleanup for scheduling metrics (#14800)
Woosuk Kwon committed over 1 year ago
[Misc] Gemma3ForConditionalGeneration supports LoRA (#14797)
Jee Jee Li committed over 1 year ago
[Bugfix] Fix small typo in the example of Streaming delimiter (#14793)
WeiCheng committed over 1 year ago
[Misc] Clean up type annotation for `SupportsMultiModal` (#14794)
Cyrus Leung committed over 1 year ago
[BugFix]Fix performance serving benchmark when enable profiling (#14737)
Li Wang committed over 1 year ago
[Bugfix][Kernel][CPU] Fix num_tokens in CPU rotary embedding kernel (#14667)
Thien Tran committed over 1 year ago
[Attention] MLA get rid of materialization (#14770)
Lucas Wilkinson committed over 1 year ago
[CI] Fix missing example model id in processor test (#14787)
Roger Wang committed over 1 year ago
[Feature] Add visionarena offline support for benchmark_throughput (#14654)
Jennifer Zhao committed over 1 year ago
[Neuron] flatten test parameterization for neuron attention kernels (#14712)
Liangfu Chen committed over 1 year ago
[Misc][Minor] Simplify `SamplingParams.__post_init__()` (#14772)
Nick Hill committed over 1 year ago
[ci] Reduce number of tests in fastcheck (#14782)
Kevin H. Luu committed over 1 year ago
Thien Tran committed over 1 year ago
[Kernel] [V1] Further optimizations to ROCm (Triton) Backend to better handle GQA. (#14431)
Thomas Parnell committed over 1 year ago
[Kernel] LoRA - Enable CUDAGraphs for V1 (#14626)
Varun Sundar Rabindranath committed over 1 year ago
[V1] Temporarily disable FlashInfer Rejection Sampler (#14788)
Woosuk Kwon committed over 1 year ago
[V1] Move OOM check into sampler run (#14728)
Roger Wang committed over 1 year ago