A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 5, 09:45 UTC
main
[ROCm] Remove unnecessary fp8 roundtrip in gather cache NHD dequant (#39122)
Andrew Barnes committed 7 months ago
[Perf] Optimize redundant sync for pooling model, 3.7% Throughput Improvement (#39113)
Wentao Ye committed 7 months ago
[Docker] Add fastsafetensors to NVIDIA Dockerfile (#38950)
Zhewen Li committed 7 months ago
[XPU] Skip VLLM_BATCH_INVARIANT for XPU in EAGLE DP test (#39164)
sihao_li committed 7 months ago
[Multimodal] Fix nested_tensors_equal: add length check for lists and tuple support (#38388)
Khairul Kabir committed 7 months ago
[Refactor] Improve indexer decode path metadata preparation (#38865)
Yongye Zhu committed 7 months ago
[PD][HeteroArch]Fix accuracy issue with CPU_ATTN as Decoder and Flash_ATTN as prefiller (#38935)
Chendi.Xue committed 7 months ago
[CI] Fix mypy for `vllm/v1/ops` (#39219)
Wentao Ye committed 7 months ago
Fix Responses JSON schema alias serialization (#38519)
noobHappylife committed 7 months ago
[Model] Update ColModernVBERT to support latest HF checkpoint (#39307)
Ilya Boytsov committed 7 months ago
[Bug] Fix routing bias dtype for trtllm per-block fp8 moe (#38989)
Wei Zhao committed 7 months ago
[Bugfix] Fix cpu-offload-gb assertion with non-default block sizes (#36461)
Ajay Anubolu committed 7 months ago
[Gemma4] Support quantized MoE (#39045)
Dipika Sikka committed 7 months ago
[UX] Integrate DeepGEMM into vLLM wheel via CMake (#37980)
Michael Goin committed 7 months ago
[XPU][UT] update UTs in CI (#39296)
liuzhenwei committed 7 months ago
Maral committed 7 months ago
[Bugfix] FlashInfer MXINT4 MoE crashes, missing do_finalize (#39315)
Benjamin Chislett committed 7 months ago
[torch.compile] Allow usage of Opaque Objects in PyTorch 2.11 (#39286)
Richard Zou committed 7 months ago
[Bugfix]Fix EP precision for Qwen3.5, Qwen3-Next (#39181)
Kai Song committed 7 months ago
[CI Bug] Fix pre-commit issue in main (#39347)
Wentao Ye committed 7 months ago