A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 23:45 UTC
main
[Refactor] Remove `moe_align_block_size_triton` (#21335)
Wentao Ye committed over 1 year ago
[BugFix] Fix shared storage connector load kv only load attention layer (#21428)
WeiQing Chen committed over 1 year ago
[Misc] Improve memory profiling debug message (#21429)
Ye (Charlotte) Qi committed over 1 year ago
[Bug] Fix `has_flashinfer_moe` Import Error when it is not installed (#21634)
Wentao Ye committed over 1 year ago
Support encoder-only models without KV-Cache (#21270)
Maximilien de Bayser committed over 1 year ago
[Bugfix] Investigate Qwen2-VL failing test (#21527)
Isotr0py committed over 1 year ago
Migrate AyaVisionImagePixelInputs to TensorSchema for shape validation (#21622)
Benji Beck committed over 1 year ago
Migrate AriaImagePixelInputs to TensorSchema for shape validation (#21620)
Benji Beck committed over 1 year ago
[Take 2] Correctly kill vLLM processes after benchmarks (#21646)
Huy Do committed over 1 year ago
[Misc] remove unused try-except in pooling config check (#21618)
Reid committed over 1 year ago
Lyu Han committed over 1 year ago
[TPU][Test] Divide TPU v1 Test into 2 parts. (#21431)
QiliangCui committed over 1 year ago
Alexandre JUAN committed over 1 year ago
Correctly kill vLLM processes after finishing serving benchmarks (#21641)
Huy Do committed over 1 year ago
[Docs] add offline serving multi-modal video input expamle Qwen2.5-VL (#21530)
WeiQing Chen committed over 1 year ago
[Model] Ultravox: Support Llama 4 and Gemma 3 backends (#17818)
Farzad Abdolhosseini committed over 1 year ago
[Feature] Add support for MoE models in the calibration-free RTN-based quantization (#20766)
Alex Kogan committed over 1 year ago
[TPU] Update ptxla nightly version to 20250724 (#21555)
Chengji Yao committed over 1 year ago
[Bugfix] Always set RAY_ADDRESS for Ray actor before spawn (#21540)
Rui Qiao committed over 1 year ago
[Bugfix] Fix sync_and_slice_intermediate_tensors (#21537)
Rui Qiao committed over 1 year ago