A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[Bugfix][CI/Build][ROCm] Make sure to use the headers from the build folder on ROCm (#22264)
Gregory Shtrasberg committed over 1 year ago
[Bugfix] Skip dead and non-GPU nodes for Ray DP engine allocation (#22275)
Rui Qiao committed over 1 year ago
[Perf] Parallelize fill_bitmask to accelerate high-throughput guided decoding (#21862)
Benjamin Chislett committed over 1 year ago
[Bugfix] Fix MoE BNB version (#22260)
Jee Jee Li committed over 1 year ago
[Bugfix] Fix 3D input passed into cutlass_scaled_mm (#22278)
Michael Goin committed over 1 year ago
[Bugfix] Remove faulty test for oot attention backend (#22286)
Michael Goin committed over 1 year ago
[CI][TPU] Fix docker clean up (#22271)
Siyuan Liu committed over 1 year ago
[bugfix] fix blackwell deepep installation (#22255)
youkaichao committed over 1 year ago
[V1] port xformers backend to v1 (#21342)
Giancarlo Delfin committed over 1 year ago
[Refactor] Remove Unused Environment Variable `VLLM_NO_DEPRECATION_WARNING` (#22199)
Wentao Ye committed over 1 year ago
[CI/Build] Update flashinfer to 0.2.9 (#22233)
Michael Goin committed over 1 year ago
Use UV_LINK_MODE=copy in Dockerfile to avoid hardlink fail (#22128)
Michael Goin committed over 1 year ago
[V0 Deprecation][TPU] Remove V1 flag check from tests (#22248)
Nicolò Lucchesi committed over 1 year ago
[Misc] correct static type check for GroupCoordinator (#21946)
Ning Xie committed over 1 year ago
[NVIDIA] Support Flashinfer TRT-LLM Prefill Attention Kernel (#22095)
elvischenv committed over 1 year ago
[Feature] Non-contiguous Support for FP8 Quantization (#21961)
Wentao Ye committed over 1 year ago
Migrate KimiVLImagePixelInputs to TensorSchema (#21769)
Benji Beck committed over 1 year ago
[Docs][TPU] Highlight TPU Software version selection (#22242)
Nicolò Lucchesi committed over 1 year ago
[Model] Pooling model activation supports per request control by PoolingParams (#20538)
wang.yuqi committed over 1 year ago
[Core] Factor out common logic for MM budget calculation (#22228)
Cyrus Leung committed over 1 year ago