A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 7, 00:15 UTC
main
[Kernel] Add fused_sigmoid_gating_delta_rule_update kernel for Qwen3 Next (#35777)
Xin Yang committed 8 months ago
[XPU] Add test script of PD disaggregation (#36434)
liuzhenwei committed 8 months ago
[Frontend] Add Support for MM Encoder/Decoder Beam Search (Online Transcriptions) (#36160)
Alex Brooks committed 8 months ago
[Bugfix] Avoid to replace non-tensor members in cpu model runner (#36430)
Li, Jiang committed 8 months ago
[Frontend][2/n] Improve pooling entrypoints | embed. (#36110)
wang.yuqi committed 8 months ago
[Bugfix] Skip out-of-stage layers in get_layers_from_vllm_config for pipeline parallel (#36243)
Tushar Shetty committed 8 months ago
feat(attention): extract KV-cache update from FlexAttention backend (#36263)
cong-or committed 8 months ago
[Misc] Move processors to `transformers_utils` (#35953)
Cyrus Leung committed 8 months ago
Increase Flexibility for OOV Multimodal Token Handling (#34858)
Alex Brooks committed 8 months ago
fix: Use iterator as not to store all the file loads in memory at once (#36149)
Shaun Kotek committed 8 months ago
[Examples][1/n] Resettle basic examples. (#35579)
wang.yuqi committed 8 months ago
[Bugfix] Fix CPU OMP autobind assertion to use local_world_size (#35815)
Weiguang Li committed 8 months ago
[Dependency] Remove default ray dependency (#36170)
Wentao Ye committed 8 months ago
Allow `markdownlint` to run locally (#36398)
Harry Mellor committed 8 months ago
[XPU][Doc] update xpu document about triton dependency/conflict issue. (#36301)
Kunshang Ji committed 8 months ago
[cudagraph] fix cudagraph warning in deepseekv32 (#28044)
Jiangyun Zhu committed 8 months ago
Add support for ModelOpt MXFP8 MoE models (#35986)
danisereb committed 8 months ago
[Frontend] Add GPU-less render serving path (`vllm launch render`) (#36166)
Sage committed 8 months ago
[Model] Nano Nemotron VL - fast media preprocessing (#35657)
nvnbagrov committed 8 months ago
[CI] fix flaky empty responses and add diagnostic assertions in vision chat tests (#36341)
Andreas Karatzas committed 8 months ago