A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 07:15 UTC
main
[BugFix] Make DP work with connector-delayed new requests (#18559)
Nick Hill committed over 1 year ago
[V1] Allocate kv_cache with stride order for V1 (#18775)
Nicolò Lucchesi committed over 1 year ago
[Misc] Remove duplicate init for self.vllm_config (#18896)
CYJiang committed over 1 year ago
[Deprecation] Disallow pos-args other than `model` when initializing `LLM` (#18802)
Cyrus Leung committed over 1 year ago
[ROCm][V0][Attention] Revert to the previous FA triton kernel (#18226)
Gregory Shtrasberg committed over 1 year ago
[Attention][V1] Toggle for v1 attention backend (#18275)
Gregory Shtrasberg committed over 1 year ago
[Bugfix] Fix the failing gte embedding test (#18720)
Isotr0py committed over 1 year ago
[Doc] Fix codeblocks formatting in LoRA adapters documentation (#18907)
Hyogeun Oh (오효근) committed over 1 year ago
[Misc][Tools][Benchmark] Add benchmark_serving supports for llama.cpp. (#18692)
Duyi-Wang committed over 1 year ago
Fix an error in dummy weight loading for quantization models (#18855)
Chenyaaang committed over 1 year ago
[BugFix] Update pydantic to fix error on python 3.10 (#18852)
Luka Govedič committed over 1 year ago
[Bugfix] Ensure tensors are contiguous during serialisation (#18860)
Lukas Geiger committed over 1 year ago
[Misc] Replace TODO in serving transcription (#18895)
Nicolò Lucchesi committed over 1 year ago
[Bugfix] Fix misleading information in the documentation (#18845)
Jee Jee Li committed over 1 year ago
Reid committed over 1 year ago
[Doc] Remove redundant spaces from compatibility_matrix.md (#18891)
Michael Yao committed over 1 year ago
[LoRA] Add LoRA support for InternVL (#18842)
Jee Jee Li committed over 1 year ago
[Neuron] Add multi-LoRA support for Neuron. (#18284)
Satyajith Chilappagari committed over 1 year ago
Fixes a dead link in nightly benchmark readme (#18856)
Brent Salisbury committed over 1 year ago
Skip device and quant Pydantic validation to make plugin device work (#18843)
Yikun Jiang committed over 1 year ago