A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 13, 14:30 UTC
main
[Build] Fix release pipeline failing annotation (#28272)
Simon Mo committed about 1 year ago
[doc] add guide about the provided PTX was compiled with an unsupported toolchain (#28305)
youkaichao committed about 1 year ago
[README] Add Arm CPUs to the list of supported targets (#28290)
Fadi Arafeh committed about 1 year ago
Revert "[PerfFix] Avoid separate thread for MP executor shm spin (#28012)" (#28289)
Nicolò Lucchesi committed about 1 year ago
[Log] update shm wait time msg (#28255)
Boyuan Feng committed about 1 year ago
[Feature] Default `ignore_eos` True for `random` dataset (#28227)
Wentao Ye committed about 1 year ago
[CLI] add --max-tokens to `vllm complete` (#28109)
Iceber Gu committed about 1 year ago
[Bugfix] Use latency MOE backend as default for Flashinfer and other misc fixes (#27439)
Pavani Majety committed about 1 year ago
[Core][MM] Add mechanism to configure multimodal fields which should stay on CPU (#28168)
Lukas Geiger committed about 1 year ago
[fix] Revert "fixing mm placeholder replacement issue with gemma3" (#28285)
Kevin H. Luu committed about 1 year ago
[Misc][Model][Refactor] Pass the prefix into Linear layers (#28259)
Mengqing Cao committed about 1 year ago
[CPU]Avoid repeated random sample compile (#28260)
Zhang Xiangze committed about 1 year ago
[FixBug]Aeala/ShareGPT_Vicuna_unfiltered marked as multimodal benchmark (#28265)
汪志鹏 committed about 1 year ago
[Perf] Introduce FlattenLogprobs to store logprobs results to reduce GC overhead (#28171)
Jialin Ouyang committed about 1 year ago
[Kernel] LoRA triton kernels support PDL (#27402)
Jee Jee Li committed about 1 year ago
Copilot committed about 1 year ago
[Bug] Fix missing token_ids for reasoning parser models in chat completions #28246 (#28256)
baonudesifeizhai committed about 1 year ago
[amd][gptoss] Perf gain because of block alignment (#28024)
smit kadvani committed about 1 year ago
Fix issues from #28242 (#28257)
Harry Mellor committed about 1 year ago
Add runai model streamer e2e test for GCS (#28079)
Alexis MacAskill committed about 1 year ago