A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 00:30 UTC
main
[Kernel] Fixup for CUTLASS kernels in CUDA graphs (#4954)
Tyler Michael Smith committed over 2 years ago
[misc] remove comments that were supposed to be removed (#4977)
SangBin Cho committed over 2 years ago
[CI/Build] Enforce style for C++ and CUDA code with `clang-format` (#4722)
Michael Goin committed over 2 years ago
[Frontend] Dynamic RoPE scaling (#4638)
sasha0552 committed over 2 years ago
[Bugfix][Kernel] Add head size check for attention backend selection (#4944)
Isotr0py committed over 2 years ago
[Bugfix] Fix flag name for `max_seq_len_to_capture` (#4935)
Kante Yin committed over 2 years ago
[CI/Build] Codespell ignore `build/` directory (#4945)
Michael Goin committed over 2 years ago
[Docs] Add acknowledgment for sponsors (#4925)
Simon Mo committed over 2 years ago
[Model] Add Phi-2 LoRA support (#4886)
Isotr0py committed over 2 years ago
[Model] add rope_scaling support for qwen2 (#4930)
HUANG Fei committed over 2 years ago
[Core] Fix scheduler considering "no LoRA" as "LoRA" (#4897)
Antoni Baum committed over 2 years ago
[Doc]Add documentation to benchmarking script when running TGI (#4920)
Kuntai Du committed over 2 years ago
[Core] Sharded State Loader download from HF (#4889)
Aurick Qiao committed over 2 years ago
[Bugfix] Fix dummy weight for fp8 (#4916)
Mor Zusman committed over 2 years ago
[Build/CI] Enabling AMD Entrypoints Test (#4834)
Alexei-V-Ivanov-AMD committed over 2 years ago
[Misc]: allow user to specify port in distributed setting (#4914)
Wenwei Zhang committed over 2 years ago
Alexander Matveev committed over 2 years ago
[Model] LLaVA model refactor (#4910)
Cyrus Leung committed over 2 years ago
[Kernel] Add flash-attn back (#4907)
Woosuk Kwon committed over 2 years ago
[Kernel] Add marlin_24 unit tests (#4901)
Alexander Matveev committed over 2 years ago