A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 19:30 UTC
main
[ROCm] Upgrade PyTorch nightly version (#6845)
Woosuk Kwon committed over 2 years ago
[Bugfix]: Fix Tensorizer test failures (#6835)
Sanger Steel committed over 2 years ago
[Bug Fix] Illegal memory access, FP8 Llama 3.1 405b (#6852)
Lucas Wilkinson committed over 2 years ago
[Frontend] Factor out code for running uvicorn (#6828)
Cyrus Leung committed over 2 years ago
[TPU] Support collective communications in XLA devices (#6813)
Woosuk Kwon committed over 2 years ago
enforce eager mode with bnb quantization temporarily (#6846)
chenqianfzh committed over 2 years ago
Gurpreet Singh Dhami committed over 2 years ago
[Doc] Update SkyPilot doc for wrong indents and instructions for update service (#4283)
Zhanghao Wu committed over 2 years ago
[Doc] Add Nemotron to supported model docs (#6843)
Michael Goin committed over 2 years ago
Li, Jiang committed over 2 years ago
[Misc][TPU] Support TPU in initialize_ray_cluster (#6812)
Woosuk Kwon committed over 2 years ago
[Build/CI][ROCm] Minor simplification to Dockerfile.rocm (#6811)
Woosuk Kwon committed over 2 years ago
[Bugfix][Kernel] Promote another index to int64_t (#6838)
Tyler Michael Smith committed over 2 years ago
[Model] Support Nemotron models (Nemotron-3, Nemotron-4, Minitron) (#6611)
Michael Goin committed over 2 years ago
[doc][debugging] add known issues for hangs (#6816)
youkaichao committed over 2 years ago
[Core] Use array to speedup padding (#6779)
Peng Guanwen committed over 2 years ago
[Bugfix] [Easy] Fixed a bug in the multiprocessing GPU executor. (#6770)
Anthony Platanios committed over 2 years ago
Fix ReplicatedLinear weight loading (#6793)
QQSong committed over 2 years ago
[ci] Mark tensorizer as soft fail and separate from grouped test (#6810)
Kevin H. Luu committed over 2 years ago
[ci][distributed] fix flaky tests (#6806)
youkaichao committed over 2 years ago