A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 23:15 UTC
main
[Model] MLPSpeculator speculative decoding support (#4947)
Joshua Rosenkranz committed over 2 years ago
[distributed][misc] use fork by default for mp (#5669)
youkaichao committed over 2 years ago
[Frontend] Add FlexibleArgumentParser to support both underscore and dash in names (#5718)
Michael Goin committed over 2 years ago
[Bugfix] Fix the CUDA version check for FP8 support in the CUTLASS kernels (#5715)
Tyler Michael Smith committed over 2 years ago
[Kernel] Update Cutlass int8 kernel configs for SM80 (#5275)
Varun Sundar Rabindranath committed over 2 years ago
[Model] Port over CLIPVisionModel for VLMs (#5591)
Roger Wang committed over 2 years ago
[Kernel] Update Cutlass int8 kernel configs for SM90 (#5514)
Varun Sundar Rabindranath committed over 2 years ago
[Bugfix][Doc] FIx Duplicate Explicit Target Name Errors (#5703)
Roger Wang committed over 2 years ago
[Misc] Improve conftest (#5681)
Cyrus Leung committed over 2 years ago
[ci] Limit num gpus if specified for A100 (#5694)
Kevin H. Luu committed over 2 years ago
Dipika Sikka committed over 2 years ago
[Doc] Update docker references (#5614)
Rafael Vasquez committed over 2 years ago
[Bugfix] AsyncLLMEngine hangs with asyncio.run (#5654)
zifeitong committed over 2 years ago
[ci][distributed] add tests for custom allreduce (#5689)
youkaichao committed over 2 years ago
[Frontend][Bugfix] Fix preemption_mode -> preemption-mode for CLI arg in arg_utils.py (#5688)
Michael Goin committed over 2 years ago
[ci] Add A100 queue into AWS CI template (#5648)
Kevin H. Luu committed over 2 years ago
[CI/Build] Add tqdm to dependencies (#5680)
Cyrus Leung committed over 2 years ago
[Misc]Add param max-model-len in benchmark_latency.py (#5629)
DearPlanet committed over 2 years ago
[Bugfix] Fix sampling_params passed incorrectly in Phi3v example (#5684)
Isotr0py committed over 2 years ago
[Model] Add FP8 kv cache for Qwen2 (#5656)
Michael Goin committed over 2 years ago