A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 01:00 UTC
main
[ROCm] warpSize is being made non constexpr in ROCm 7.0 (#20330)
Gregory Shtrasberg committed over 1 year ago
Add full serve CLI reference back to docs (#20978)
Harry Mellor committed over 1 year ago
[Deprecation] Remove `nullable_kvs` (#20969)
Harry Mellor committed over 1 year ago
Harry Mellor committed over 1 year ago
[Deprecation] Remove everything scheduled for removal in v0.10.0 (#20979)
Harry Mellor committed over 1 year ago
[CI/Build] Fix wrong path in Transformers Nightly Models Test (#20994)
Cyrus Leung committed over 1 year ago
Patrick von Platen committed over 1 year ago
[v1][core] Support for attention free models (#20811)
Christian Pinto committed over 1 year ago
[Deprecation] Remove `TokenizerPoolConfig` (#20968)
Harry Mellor committed over 1 year ago
[doc] Add more details for Ray-based DP (#20948)
Rui Qiao committed over 1 year ago
[MISC] Add init files for python package (#20908)
Li Wang committed over 1 year ago
[V1] [Hybrid] Refactor mamba state shape calculation; enable V1 via cli (#20840)
Thomas Parnell committed over 1 year ago
[TPU] Optimize kv cache update kernel (#20415)
Yifei Teng committed over 1 year ago
[Docs] Improve documentation for ray cluster launcher helper script (#20602)
Ricardo Decal committed over 1 year ago
[frontend] Refactor CLI Args for a better modular integration (#20206)
kourosh hakhamaneshi committed over 1 year ago
[Docs] Improve documentation for RLHF example (#20598)
Ricardo Decal committed over 1 year ago
[frontend] Add --help=page option for paginated help output (#20961)
Reid committed over 1 year ago
[Misc] Refactor AllReduceFusionPass. Remove parameter (#20918)
Ilya Markov committed over 1 year ago
Implement Async Scheduling (#19970)
Woosuk Kwon committed over 1 year ago
[Model] Add AutoWeightsLoader support for BERT, RoBERTa (#20534)
Jennifer He committed over 1 year ago