A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 11:00 UTC
main
[Hardware/NVIDIA/Modelopt] Fix modelopt forward method for v1 torch.compile (#18101)
Pavani Majety committed over 1 year ago
[Fix] Support CUDAGraph capture for encoder-decoder on ROCm (#18104)
Luka Govedič committed over 1 year ago
[v1][KVCacheManager] pass num_new_computed_tokens to kv cache manager (#18001)
Chen Zhang committed over 1 year ago
[FEAT] [ROCm]: Add AITER Block-Scaled GEMM Feature (#14968)
vllmellm committed over 1 year ago
[Docs] Expand security doc with firewall info (#18081)
Russell Bryant committed over 1 year ago
[V1] DP scale-out (2/N): Decouple engine process management and comms (#15977)
Nick Hill committed over 1 year ago
Update deprecated type hinting in `vllm/adapter_commons` (#18073)
Harry Mellor committed over 1 year ago
Update deprecated type hinting in `vllm/compilation` (#18072)
Harry Mellor committed over 1 year ago
Give auto-merge label workflow permission to add labels to issues (#18078)
Harry Mellor committed over 1 year ago
Convert `benchmarks` to `ruff format` (#18068)
Harry Mellor committed over 1 year ago
[Bugfix] Fix entrypoints metrics tests (#18063)
Cyrus Leung committed over 1 year ago
[CI] set token permissions for pre-commit CI job (#17729)
Russell Bryant committed over 1 year ago
[CI] Add token permissions for add-ready-label CI job (#17730)
Russell Bryant committed over 1 year ago
[CI] Add workflow permissions for helm CI job (#17727)
Russell Bryant committed over 1 year ago
[CI] Set token permissions for reminder comment CI job (#17728)
Russell Bryant committed over 1 year ago
Update deprecated type hinting in `vllm/transformers_utils` (#18058)
Harry Mellor committed over 1 year ago
Update deprecated type hinting in `vllm/profiler` (#18057)
Harry Mellor committed over 1 year ago
Update deprecated type hinting in `model_executor/layers` (#18056)
Harry Mellor committed over 1 year ago
[doc] add download/list/delete HF model CLI usage (#17940)
Reid committed over 1 year ago
[Fix] check to make sure processor has chat templates (#18047)
Aaron Pham committed over 1 year ago