A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 1, 22:30 UTC
main
[ROCm] Bump fastsafetensors to v0.3.2 from PyPI, remove git source build (#43625)
Turner Jabbour committed 5 months ago
[PD][Nixl] Mamba prefix caching mode support (#42554)
Nicolò Lucchesi committed 5 months ago
[Bugfix][Compile] Guard per_token_group_fp8_quant lookup on non-CUDA platforms (#44476)
QiliangCui2023 committed 5 months ago
Add GH token to docs build pre run check (#44534)
Harry Mellor committed 5 months ago
Refactor CT NVFP4 linear to use a single class (#42443)
Dipika Sikka committed 5 months ago
[Bugfix] fix EVS for qwen3-vl (#44205)
Rui "Garry" Gao committed 5 months ago
[Attention] Mamba attention module refactor - LINEAR (#43556)
wangxiyuan committed 5 months ago
[perf] Add gemma RMS AR fusion (#42646)
Jiahan Chang (Cyrus) committed 5 months ago
Fix Kimi-K2.5 FlashInfer ViT metadata (#44493)
Kevin_Xiong committed 5 months ago
Wei Zhao committed 5 months ago
[CI] Reverted gitignore changes (#44497)
Andreas Karatzas committed 5 months ago
[Frontend] Consolidate online serving utils. (#44479)
wang.yuqi committed 5 months ago
[Inductor] Fast-path Inductor fallback for vllm::*/vllm_aiter::* custom ops (#42129)
Oxana Korzh committed 5 months ago
[CI] Resolve release V2 docker build after ROCm CI wheels change (#44463)
Andreas Karatzas committed 5 months ago
[EPLB] Nixl communicator optimization. Zero-copy transfers (#41633)
Ilya Markov committed 5 months ago
optimize the compressor 128 split cutedsl kernel (#44230)
Jie Fang committed 5 months ago
[Refactor] Remove dead code in tests and parallel_state (#41471)
Wentao Ye committed 5 months ago
[MM][Perf][CG] Support ViT full CUDA graph for InternVL (#41759)
Oğuzhan KIR committed 5 months ago
[KV Connector] Update lmcache kv_offloading_backend to use LMCacheMPConnector (#42865)
maobaolong committed 5 months ago
[Bugfix] Fix VLLMNotFoundError when using LoRA adapter name in poolin… (#44410)
Ted Mostly committed 5 months ago