A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 09:30 UTC
main
[CI/Build] Add bot to close stale issues and PRs (#9436)
Russell Bryant committed about 2 years ago
[Model] Add Qwen2-Audio model support (#9248)
Yunfei Chu committed about 2 years ago
[Frontend] Enable Online Multi-image Support for MLlama (#9393)
Alex Brooks committed about 2 years ago
[torch.compile] Adding torch compile annotations to some models (#9614)
Yongzao committed about 2 years ago
[Misc] Separate total and output tokens in benchmark_throughput.py (#8914)
Michael Goin committed about 2 years ago
[Bugfix] Fix divide by zero when serving Mamba models (#9617)
Tyler Michael Smith committed about 2 years ago
[misc] comment to avoid future confusion about baichuan (#9620)
youkaichao committed about 2 years ago
[Bugfix] Fix `_init_vision_model` in NVLM_D model (#9611)
Cyrus Leung committed about 2 years ago
[Model] Add min_pixels / max_pixels to Qwen2VL as mm_processor_kwargs (#9612)
Alex Brooks committed about 2 years ago
[VLM] Enable overriding whether post layernorm is used in vision encoder + fix quant args (#9217)
Cyrus Leung committed about 2 years ago
[Model] Initialize Florence-2 language backbone support (#9555)
Isotr0py committed about 2 years ago
[Hardware][XPU] using current_platform.is_xpu (#9605)
Mengqing Cao committed about 2 years ago
[Build] Fix `FetchContent` multiple build issue (#9596)
Luka Govedič committed about 2 years ago
Cyrus Leung committed about 2 years ago
Flex Wang committed about 2 years ago
[Bugfix] Generate exactly input_len tokens in benchmark_throughput (#9592)
Chen Zhang committed about 2 years ago
[Doc]: Update tensorizer docs to include vllm[tensorizer] (#7889)
Seth Kimmel committed about 2 years ago
[BugFix] Fix metrics error for --num-scheduler-steps > 1 (#8234)
yulei committed about 2 years ago
[Bugfix] Fix spurious "No compiled cutlass_scaled_mm ..." for W8A8 on Turing (#9487)
Lucas Wilkinson committed about 2 years ago
[Misc] Make benchmarks use EngineArgs (#9529)
Jeremy Arnold committed about 2 years ago