A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 00:45 UTC
main
[V1][Bugfix] Fix assertion when mm hashing is turned off (#12439)
Roger Wang committed almost 2 years ago
[V1][Perf] Reduce scheduling overhead in model runner after cuda sync (#12094)
Keyun Tong committed almost 2 years ago
[Misc][Bugfix] FA3 support to ViT MHA layer (#12435)
Roger Wang committed almost 2 years ago
[TPU][CI] Update torchxla version in requirement-tpu.txt (#12422)
Siyuan Liu committed almost 2 years ago
[Misc] Add FA2 support to ViT MHA layer (#12355)
Isotr0py committed almost 2 years ago
[ROCm][MoE] MI300 tuned configs Mixtral-8x(7B,22B) | fp16, fp8 (#12408)
Divakar Verma committed almost 2 years ago
[Bugfix] Fix BLIP-2 processing (#12412)
Cyrus Leung committed almost 2 years ago
[Bugfix][Kernel] Fix moe align block issue for mixtral (#12413)
ElizaWszola committed almost 2 years ago
Lucas Wilkinson committed almost 2 years ago
[Misc] Remove deprecated code (#12383)
Cyrus Leung committed almost 2 years ago
[Bugfix][Kernel] Fix CUDA 11.8 being broken by FA3 build (#12375)
Lucas Wilkinson committed almost 2 years ago
[Misc] Enable proxy support in benchmark script (#12356)
Junichi Sato committed almost 2 years ago
[ci/build] sync default value for wheel size (#12398)
youkaichao committed almost 2 years ago
[Hardware][Gaudi][Doc] Add missing step in setup instructions (#12382)
Mohit Deopujari committed almost 2 years ago
[ci/build] fix wheel size check (#12396)
youkaichao committed almost 2 years ago
[Misc] Use VisionArena Dataset for VLM Benchmarking (#12389)
Roger Wang committed almost 2 years ago
[perf] fix perf regression from #12253 (#12380)
youkaichao committed almost 2 years ago
[V1] Increase default batch size for H100/H200 (#12369)
Woosuk Kwon committed almost 2 years ago
Update compressed-tensors version (#12367)
Dipika Sikka committed almost 2 years ago
[Bugfix] Path join when building local path for S3 clone (#12353)
omer-dayan committed almost 2 years ago