A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 22:00 UTC
main
Build some nightly wheels by default (#6380)
Simon Mo committed over 2 years ago
[MISC] Upgrade dependency to PyTorch 2.3.1 (#5327)
Cody Yu committed over 2 years ago
[ci] Add GHA workflows to enable full CI run (#6381)
Kevin H. Luu committed over 2 years ago
[ci] Add grouped tests & mark tests to run by default for fastcheck pipeline (#6365)
Kevin H. Luu committed over 2 years ago
[Misc] add fixture to guided processor tests (#6341)
Yihuan Bu committed over 2 years ago
[Build/CI] Checking/Waiting for the GPU's clean state (#6379)
Alexei-V-Ivanov-AMD committed over 2 years ago
[Bugfix] Fix dtype mismatch in PaliGemma (#6367)
Cyrus Leung committed over 2 years ago
[ Misc ] Support Models With Bias in `compressed-tensors` integration (#6356)
Robert Shaw committed over 2 years ago
[Misc][Bugfix] Update transformers for tokenizer issue (#6364)
Roger Wang committed over 2 years ago
[ Misc ] Remove separate bias add (#6353)
Robert Shaw committed over 2 years ago
[ROCm][AMD] unify CUDA_VISIBLE_DEVICES usage in cuda/rocm (#6352)
Hongxia Yang committed over 2 years ago
[CI/Build] (2/2) Switching AMD CI to store images in Docker Hub (#6350)
adityagoel14 committed over 2 years ago
[Model][Phi3-Small] Remove scipy from blocksparse_attention (#6343)
Michael Goin committed over 2 years ago
[Bugfix] Fix usage stats logging exception warning with OpenVINO (#6349)
Helena Kloosterman committed over 2 years ago
[distributed][misc] be consistent with pytorch for libcudart.so (#6346)
youkaichao committed over 2 years ago
[Misc] Remove flashinfer warning, add flashinfer tests to CI (#6351)
Lily Liu committed over 2 years ago
[ BugFix ] Prompt Logprobs Detokenization (#6223)
Robert Shaw committed over 2 years ago
[CI/Build] Add nightly benchmarking for tgi, tensorrt-llm and lmdeploy (#5362)
Kuntai Du committed over 2 years ago
[doc] update pipeline parallel in readme (#6347)
youkaichao committed over 2 years ago
[bug fix] Fix llava next feature size calculation. (#6339)
xwjiang2010 committed over 2 years ago