A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 2, 03:30 UTC
main
Bump actions/github-script from 8.0.0 to 9.0.0 (#39667)
dependabot[bot] committed 5 months ago
[MRV2] Also enable MRV2 for Llama and Mistral dense models (#43458)
Nick Hill committed 5 months ago
[Anthropic] Support system role messages inside messages array (#44283)
Chauncey committed 5 months ago
[Feature] Support EPLB for DeepSeek v4 Mega Moe (#43339)
Wei Zhao committed 5 months ago
[Bugfix] flashinfer: fail fast when --kv-cache-dtype nvfp4 used on unsupported arch (#43669)
Kartavya sonar committed 5 months ago
[BugFix] Fix Humming MoE deploy error (#43100)
Alireza Dadgarnia committed 5 months ago
[XPU] Enable rms_norm/act quant fusions (#43963)
liuzhenwei committed 5 months ago
[Refactor] Remove dead code from parser infrastructure (#44279)
Flora Feng committed 5 months ago
[Core] Move `max_concurrent_batches` to `VllmConfig` (#44274)
Nick Hill committed 5 months ago
[compressed-tensors] Asymmetric support for MoE WNA16 marlin (#44025)
Brian Dellabetta committed 5 months ago
[Misc] Support local image encoding in benchmarks (#43843)
XiaoZ committed 5 months ago
Migrate header files to torch stable abi (#44013)
Chris Leonard committed 5 months ago
[Rust Frontend] Cover different thinking modes in roundtrip tests (#44320)
Bugen Zhao committed 5 months ago
[ROCm] Fix AITER RMSNormQuantFusion for Kimi-Linear (#44308)
pschlan-amd committed 5 months ago
[Rust Frontend] Support recursive tool parameter conversion (#44299)
Bugen Zhao committed 5 months ago
[XPU] [Bug] remove xpuw4a16 output size check (#44168)
zofia committed 5 months ago
[BugFix] [GDN] Read linear_key_head_dim from hf_text_config for multimodal models (#43978)
IdoAtadTD committed 5 months ago
[FlashAttention] Sync FA with upstream (#44065)
Matthew Bonanni committed 5 months ago
[Bugfix] Vendor MiniCPMV/MiniCPMO processors to unblock Transformers v5 (#44282)
王金旭 committed 5 months ago
Support ModelOpt MXFP8 non-gated MoE (#42958)
TomerBN-Nvidia committed 5 months ago