A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 23:15 UTC
main
[misc][distributed] use 127.0.0.1 for single-node (#5619)
youkaichao committed over 2 years ago
Hongxia Yang committed over 2 years ago
[Bugfix] Added test for sampling repetition penalty bug. (#5659)
Thomas Parnell committed over 2 years ago
[Bugfix] Fix Phi-3 Long RoPE scaling implementation (#5628)
Shukant Pal committed over 2 years ago
[Bugfix] Fix w8a8 benchmarks for int8 case (#5643)
Tyler Michael Smith committed over 2 years ago
[Bugfix] Fix CUDA version check for mma warning suppression (#5642)
Tyler Michael Smith committed over 2 years ago
[Doc] Added cerebrium as Integration option (#5553)
milo157 committed over 2 years ago
[Bugfix] Fix for inconsistent behaviour related to sampling and repetition penalties (#5639)
Thomas Parnell committed over 2 years ago
[Model] LoRA support added for command-r (#5178)
sergey-tinkoff committed over 2 years ago
[ci] Setup Release pipeline and build release wheels with cache (#5610)
Kevin H. Luu committed over 2 years ago
Dipika Sikka committed over 2 years ago
[Misc] Add OpenTelemetry support (#4687)
Ronen Schaffer committed over 2 years ago
[ci] Deprecate original CI template (#5624)
Kevin H. Luu committed over 2 years ago
[CI/Build][Misc] Update Pytest Marker for VLMs (#5623)
Roger Wang committed over 2 years ago
[Misc] Remove import from transformers logging (#5625)
Chang Su committed over 2 years ago
[bugfix][distributed] improve p2p capability test (#5612)
youkaichao committed over 2 years ago
[CI] Avoid naming different metrics with the same name in performance benchmark (#5615)
Kuntai Du committed over 2 years ago
Cyrus Leung committed over 2 years ago
youkaichao committed over 2 years ago
[Kernel] Add punica dimensions for Granite 13b (#5559)
Joe Runde committed over 2 years ago