A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 02:15 UTC
main
[Misc] Clean up Aimv2 config registration in Ovis config (#20921)
Isotr0py committed over 1 year ago
[CI] Update codeowner for compilation code (#20929)
Lu Fang committed over 1 year ago
[CI] cc folks on changes to vllm/compilation (#20925)
Richard Zou committed over 1 year ago
[Model] Add Ling implementation (#20680)
ant-yy committed over 1 year ago
[Misc] Log the reason for falling back to FlexAttention (#20699)
Cyrus Leung committed over 1 year ago
[Bugfix]: Fix messy code when using logprobs (#20910)
Chauncey committed over 1 year ago
[Misc] Remove unused function (#20909)
Reid committed over 1 year ago
[Bugfix] Bump up mistral_common to support v13 tokenizer (#20905)
22quinn committed over 1 year ago
[CI/Build] Fix OOM issue in Jina-VL test (#20907)
Cyrus Leung committed over 1 year ago
[MISC] Move bind_kv_cache to worker module (#20900)
wangxiyuan committed over 1 year ago
[Refactor][V1] Move outlines utils for V1 imports (#20878)
Aaron Pham committed over 1 year ago
[Quantization] add BNB for MixtralForCausalLM (#20893)
Jee Jee Li committed over 1 year ago
[ROCm] [Bugfix] [Critical]: Fix mamba compilation bug (#20883)
TJian committed over 1 year ago
Fix: Add missing EOFError handling in CLI complete command (#20896)
Reid committed over 1 year ago
Removing redundant python version check (#20888)
Daniel song committed over 1 year ago
[Prefix Cache] Add reproducible prefix-cache block hashing using SHA-256 + CBOR (64bit) (#20511)
Maroon Ayoub committed over 1 year ago
[Core] Add `update_config` RPC method (#20095)
22quinn committed over 1 year ago
[V1] Hybrid allocator without prefix caching (#20661)
nopperl committed over 1 year ago
[V1] [ROCm] [AITER] Upgrade AITER to commit `916bf3c` and bugfix APIs (#20880)
TJian committed over 1 year ago
[Bugfix] fix define of RerankDocument (#20877)
Liuchenlong committed over 1 year ago