A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 2, 18:15 UTC
main
[Perf] Optimize Fp8BlockScaledMMLinearKernel input_scale tensor using new_empty() (#43677)
Xin Yang committed 5 months ago
Andreas Karatzas committed 5 months ago
[Rust Frontend] Add reasoning/tool parser & renderer roundtrip tests (#43582)
Bugen Zhao committed 5 months ago
[Bugfix] Split attention groups by num_heads_q for spec-decode drafts (#43543)
Luciano Martins committed 5 months ago
[ci] Add arm64 ci image (#41303)
Kevin H. Luu committed 5 months ago
Vadim Gimpelson committed 5 months ago
[KV Connector] MooncakeStore: drop dead discard_partial_chunks parameter (#43627)
Zhewen Li committed 5 months ago
[Kernel] Porting fuse_minimax_qk_norm to manual fusion (#43410)
Jee Jee Li committed 5 months ago
[CI] Soft-fail AMD entrypoints mirror tests (#43709)
Kevin H. Luu committed 5 months ago
[DSv4] Drop _get_compressed_kv_buffer in DeepseekCompressor (#43690)
Woosuk Kwon committed 5 months ago
[Doc] Add line limit to AGENTS.md (#43635)
Woosuk Kwon committed 5 months ago
Add LM head quantization support for ModelOpt (#42124)
Wei-Ming Chen committed 5 months ago
[ROCm] Remove MegaMoE integration in deepseek v4 (#43629)
Woosuk Kwon committed 5 months ago
[MoE Refactor] W4a8 int8 oracle (#42789)
bnellnm committed 5 months ago
[MoE Refactor] Migrate ModelOptMxFp8FusedMoE to oracle (#42768)
bnellnm committed 5 months ago
[Feat][DSV4] Fuse q pad into deepseek v4 fused kernel (#43162)
Yongye Zhu committed 5 months ago
[ROCm][CI] Extend ROCm quick reduce coverage (#40990)
Andreas Karatzas committed 5 months ago
[Docs][ROCm] MoRI-IO Connector Usage Guide (#43603)
Simon Danielsson committed 5 months ago
Fix CuPy runtime deps and restore humming (#43530)
Mohammad Miadh Angkad committed 5 months ago
[XPU] Fix fused MoE LoRA kernel crash on XPU by using platform-agnos num_compute_units (#43646)
Chaojun Zhang committed 5 months ago