A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 04:15 UTC
main
Bump up version to v0.3.2 (#2968)
Zhuohan Li committed over 2 years ago
Support per-request seed (#2514)
Nick Hill committed over 2 years ago
[ROCm] Upgrade transformers to v4.38.0 (#2967)
Woosuk Kwon committed over 2 years ago
[FIX] Add Gemma model to the doc (#2966)
Zhuohan Li committed over 2 years ago
Upgrade transformers to v4.38.0 (#2965)
Woosuk Kwon committed over 2 years ago
Xiang Xu committed over 2 years ago
Add metrics to RequestOutput (#2876)
Antoni Baum committed over 2 years ago
Make vLLM logging formatting optional (#2877)
Antoni Baum committed over 2 years ago
[FIX] Fix beam search test (#2930)
Zhuohan Li committed over 2 years ago
[ROCm] include gfx908 as supported (#2792)
James Whedbee committed over 2 years ago
Fix `vllm:prompt_tokens_total` metric calculation (#2869)
Ronen Schaffer committed over 2 years ago
Add warning to prevent changes to benchmark api server (#2858)
Simon Mo committed over 2 years ago
Isotr0py committed over 2 years ago
[Test] Add basic correctness test (#2908)
Zhuohan Li committed over 2 years ago
[Minor] Small fix to make distributed init logic in worker looks cleaner (#2905)
Zhuohan Li committed over 2 years ago
Add code-revision config argument for Hugging Face Hub (#2892)
Mark Mozolewski committed over 2 years ago
multi-LoRA as extra models in OpenAI server (#2775)
jvmncs committed over 2 years ago
Defensively copy `sampling_params` (#2881)
Nick Hill committed over 2 years ago
Woosuk Kwon committed over 2 years ago
Prefix Caching- fix t4 triton error (#2517)
shiyi.c_98 committed over 2 years ago