A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 12:15 UTC
main
[Bugfix] Fix lint error (#17547)
Cyrus Leung committed over 1 year ago
[Misc]add configurable cuda graph size (#17201)
Chen Xia committed over 1 year ago
[ROCm] update installation guide to include build aiter from source instructions (#17542)
Hongxia Yang committed over 1 year ago
[Model] Refactor Ovis2 to support original tokenizer (#17537)
Isotr0py committed over 1 year ago
Move the last arguments in `arg_utils.py` to be in their final groups (#17531)
Harry Mellor committed over 1 year ago
[CI/Build] Remove `awscli` dependency (#17532)
Cyrus Leung committed over 1 year ago
[Misc] refactor example - cpu_offload_lmcache (#17460)
Reid committed over 1 year ago
Sage Moore committed over 1 year ago
[ROCm] remove unsupported archs from rocm triton flash-attention supported list (#17536)
Hongxia Yang committed over 1 year ago
[Feature][Frontend]: Deprecate --enable-reasoning (#17452)
Chauncey committed over 1 year ago
[FEAT] [ROCm]: Add Qwen/Qwen3-235B-A22B-FP8 TP4 triton fused moe config (#17535)
TJian committed over 1 year ago
[doc] add streamlit integration (#17522)
Reid committed over 1 year ago
Fix more broken speculative decode tests (#17450)
Huy Do committed over 1 year ago
[FEAT] [ROCm]: Add Qwen/Qwen3-30B-A3B-FP8 fused moe config for MI300X (#17530)
TJian committed over 1 year ago
[Misc] Clean up test docstrings and names (#17521)
Cyrus Leung committed over 1 year ago
[Frontend] Show progress bar for adding requests (#17525)
Cyrus Leung committed over 1 year ago
[Bugfix][Benchmarks] Allow benchmark of deepspeed-mii backend to select a model (#17285)
Teruaki Ishizaki committed over 1 year ago
Improve configs - `ObservabilityConfig` (#17453)
Harry Mellor committed over 1 year ago
[Misc] Optimize the Qwen3_ReasoningParser extract_reasoning_content (#17515)
Chauncey committed over 1 year ago
[Core] Enable IPv6 with vllm.utils.make_zmq_socket() (#16506)
Russell Bryant committed over 1 year ago