A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 5, 17:15 UTC
main
[torch.compile] Refactor Attention Quant Fusion Pass and Remove Boilerplate (#37373)
BadrBasowid committed 7 months ago
[CI/Build] Resolve a dependency deadlock when installing the test dependencies used in CI (#37766)
Run Yu committed 7 months ago
[4/n] Migrate FP4/W4A8 CUTLASS kernels to torch stable ABI (#37503)
mikaylagawarecki committed 7 months ago
[Quantization][Autoround][XPU] Add `W4A16` Support (#37986)
Yi Liu committed 7 months ago
[Bugfix] Fix FusedMoE weight loading with padded hidden dimensions (#37010)
SandishKumarHN committed 7 months ago
[Fix] handle PaddleOCR-VL image processor max_pixels across Transformers v4/v5 (#38629)
zhang-prog committed 7 months ago
[Bugfix] clamp dA_cumsum differences to prevent Inf in Mamba2 SSD kernels (#37501)
Jingu Kang committed 7 months ago
[Bugfix][Async] Fix async spec decoding with hybrid models (#38556)
Matthew Bonanni committed 7 months ago
[Docs] PD with Nixl compat matrix (#38628)
Nicolò Lucchesi committed 7 months ago
[Online Quant] [QeRL] Minor code cleanup (#38574)
Kyle Sayers committed 7 months ago
replace cuda_device_count_stateless() to current_platform.device_count() (#37841)
wliao2 committed 7 months ago
[CI] Avoid concurrent docker pull in intel XPU CI runners to prevent rate limit issues (#38594)
wenjun liu committed 7 months ago
[Bugfix][CI] Skip flaky `test_eagle` test (#38566)
Nicolò Lucchesi committed 7 months ago
Fix MLA runs when use_inductor_graph_partition=True (#38631)
ElizaWszola committed 7 months ago
[CI] fix LM Eval Qwen3.5 Models (B200) (#38632)
Jiangyun Zhu committed 7 months ago
[XPU]move testing dependencies from Dockerfile to xpu-test.in (#38596)
sihao_li committed 7 months ago
[EPLB] Add alternative communication for EPLB weight exchange (#33176)
Ilya Markov committed 7 months ago
[EPD] update EPD script arguments (#36742)
liuzhenwei committed 7 months ago
[CI Failure] pin colmodernvbert revision (#38612)
wang.yuqi committed 7 months ago
Fix document of torchrun_example.py (#31113)
Maosheng Liao committed 7 months ago