A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 12, 06:00 UTC
main
[ROCm] Fallback pytorch GELU with tanh approximation to GELU() (#29244)
Divakar Verma committed 12 months ago
SM120 / NVFP4: add device guard and runtime SM dispatch to cutlass_scaled_fp4_mm (#29711)
Hendrik Holtmann committed 12 months ago
[BugFix] Preserve spec decoding uniform decode when scheduling (#29759)
Nick Hill committed 12 months ago
Updated CI mirror 2025-11-25 (#29434)
Alexei-V-Ivanov-AMD committed 12 months ago
Revert #29787 and #29690 (#29815)
Kevin H. Luu committed 12 months ago
[Ascend]: Fixed the issue where OOT Platform vllm-ascend could not enable SP in Eager mode (#28935)
Nengjun Ma committed 12 months ago
Update FAQ on interleaving sliding windows support (#29796)
Finbarr Timbers committed 12 months ago
[Core][Observability] Add KV cache residency metrics (#27793)
shivampr committed 12 months ago
[ci] Make distributed 8 gpus test optional (#29801)
Kevin H. Luu committed 12 months ago
[Bugfix] Missing cached item in the MultiModalReceiverCache (#28525)
knlnguyen1802 committed 12 months ago
[Hardware][AMD] Remove ROCm skip conditions for transformers backend tests (#29782)
BADAOUI Abdennacer committed 12 months ago
[Frontend] add 'verbose_json' and 'timestamp' feature on Whisper Transcription/Translation (#24209)
sangbumlikeagod committed 12 months ago
[Doc] Update description disable_any_whitespace (#29784)
FredericOdermatt committed 12 months ago
[CI] fix url-encoding behavior in nightly metadata generation (#29787)
Shengqi Chen committed 12 months ago
[Doc] fix heading levels (#29783)
Liu Jinyi committed 12 months ago
[CI] Renovation of nightly wheel build & generation (#29690)
Shengqi Chen committed 12 months ago
[Bugfix] TypeError: 'NoneType' object is not callable (#29414)
Marcin Ostrowski committed 12 months ago
[v1] Add real sliding window calculation to FlexAttention direct BlockMask building (#26015)
Isotr0py committed 12 months ago
[multimodal][test] Reduce memory utilization for test_siglip to avoid OOM (#29504)
Zhengxu Chen committed 12 months ago
[XPU] Fix AWQ skipped layer detection in IPEX quantization (#29774)
Fanli Lin committed 12 months ago