A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[CI/Build] fix doc build warning: Failed to get 'name: description' pair (#25733)
yitingdc committed about 1 year ago
EVS Support (Video tokens pruning) (#22980)
Eugene Khvedchenya committed about 1 year ago
[Misc] Remove unnecessary memoryviews in shm_broadcast.py (#25721)
Nick Hill committed about 1 year ago
[Core] Force PIECEWISE CUDAGraph mode for encoder-decoder (#25701)
Russell Bryant committed about 1 year ago
Llamas 3.1 405B fp4 changes upstreaming from 355_wip (#25135)
Aleksandr Malyshev committed about 1 year ago
[Misc] Don't log shm dequeue delay warning on worker side (#25720)
Nick Hill committed about 1 year ago
[Refactor] Remove DeepGEMM OP Register (#25710)
Wentao Ye committed about 1 year ago
Fix routing_bias dtype (#25711)
Shu Wang committed about 1 year ago
[Model] rename NemotronH_Nano_VL -> NemotronH_Nano_VL_V2 (#25708)
tomeras91 committed about 1 year ago
[Core] Enable command line logging for LLMEngine (#25610)
Zhuohan Li committed about 1 year ago
[Spec Decode] Add Batch Parallel Ngram. Upto 8x lower overhead. (#24986)
Ekagra Ranjan committed about 1 year ago
[Optimization] Use a cheaper cache key in `get_model_architecture` (#25682)
Cyrus Leung committed about 1 year ago
[Optimization] Streamline `InputPreprocessor` (#25702)
Cyrus Leung committed about 1 year ago
[Misc] Simplify `test_argsort_mm_positions` (#25690)
Cyrus Leung committed about 1 year ago
[V0 deprecation] Clean up LoRA (#25686)
Jee Jee Li committed about 1 year ago
[V0 deprecation] Remove _VLLM_V1 suffixes from attention backend names (#25489)
Matthew Bonanni committed about 1 year ago
[V0 deprecation] Clean up V0 fallback in compilation config (#25675)
Isotr0py committed about 1 year ago
[Model] Define `merge_by_field_config` MM interface (#25676)
Cyrus Leung committed about 1 year ago
[Model] Add optional parameter to reasoning parser constructor (#25554)
Tao Hui committed about 1 year ago
[BugFix] Fix DBO hang (#25625)
Lucas Wilkinson committed about 1 year ago