A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 04:15 UTC
main
[ROCm] Dockerfile fix for flash-attention build (#2885)
Hongxia Yang committed almost 3 years ago
Philipp Moritz committed almost 3 years ago
[BugFix] Fix GC bug for `LLM` class (#2882)
Woosuk Kwon committed almost 3 years ago
Align LoRA code between Mistral and Mixtral (fixes #2875) (#2880)
Philipp Moritz committed almost 3 years ago
Don't use cupy NCCL for AMD backends (#2855)
Woosuk Kwon committed almost 3 years ago
Migrate AquilaForCausalLM to LlamaForCausalLM (#2867)
Roy committed almost 3 years ago
Fix docker python version (#2845)
Nikola Borisov committed almost 3 years ago
[Fix] Fix memory profiling when GPU is used by multiple processes (#2863)
Woosuk Kwon committed almost 3 years ago
Fix internlm after https://github.com/vllm-project/vllm/pull/2860 (#2861)
Philipp Moritz committed almost 3 years ago
Migrate InternLMForCausalLM to LlamaForCausalLM (#2860)
Philipp Moritz committed almost 3 years ago
Add LoRA support for Mixtral (#2831)
Terry committed almost 3 years ago
Remove Yi model definition, please use `LlamaForCausalLM` instead (#2854)
Philipp Moritz committed almost 3 years ago
Use CuPy for CUDA graphs (#2811)
Woosuk Kwon committed almost 3 years ago
Revert "Refactor llama family models (#2637)" (#2851)
Philipp Moritz committed almost 3 years ago
Refactor llama family models (#2637)
Roy committed almost 3 years ago
[CI] Ensure documentation build is checked in CI (#2842)
Simon Mo committed almost 3 years ago
Serving Benchmark Refactoring (#2433)
Roger Wang committed almost 3 years ago
Refactor 2 awq gemm kernels into m16nXk32 (#2723)
Rex committed almost 3 years ago
Add documentation section about LoRA (#2834)
Philipp Moritz committed almost 3 years ago
[ROCm] support Radeon™ 7900 series (gfx1100) without using flash-attention (#2768)
Hongxia Yang committed almost 3 years ago