A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 4, 17:30 UTC
main
[ROCm] Add gfx1102/gfx1103 support (#40037)
Matthias Gehre committed 7 months ago
[UT][Hardware] let torchrun example tests use the default backend (#39879)
liuzhenwei committed 7 months ago
Shengqi Chen committed 7 months ago
[Misc] Support Human-readable (k/K/m/M..) json cli arg (#40473)
Nicolò Lucchesi committed 7 months ago
[MM][CG] Support `--enable-vit-cuda-graph` option for VLM examples (#40580)
Shanshan Shen committed 7 months ago
[Responses] Add tool_choice/tools validation to match OpenAI behavior (#40399)
Flora Feng committed 7 months ago
[compile] Skip FX graph deserialiaztion on loading, further reducing warm compile time. (#40151)
Zhengxu Chen committed 7 months ago
[MoE refactor] refactor GPTQMarlinMoEMethod with MK (#37990)
Kunshang Ji committed 7 months ago
[BugFix]fix Qwen3 MoE call gate twice (#40664)
Kunshang Ji committed 7 months ago
[TurboQuant] enable FA3/FA4 for prefill paths (#40092)
Julian Huang committed 7 months ago
[XPU] disable fusion pattern support on XPU platform (#39789)
Chaojun Zhang committed 7 months ago
[BE] Fix compile time message to be consistent (use monitoring) (#40641)
Lucas Kabela committed 7 months ago
[Fix][MoRI] Align MoRI-IO message format with P2pNcclConnector and vllm-router (#39565)
Simon Danielsson committed 7 months ago
[Bugfix] Fix RMS norm + quant fusion on DeepGEMM UE8M0 path for B200 (#40552)
Lucas Kabela committed 7 months ago
[gRPC] Add standard gRPC health checking (grpc.health.v1) for Kubernetes native probes (#38016)
Honglin Cao committed 7 months ago
[Bugfix][Torch 2.12] Fix batch_invariant test with allow_override for torch 2.12 upgrade (#40562)
Lucas Kabela committed 7 months ago
[Refactor] Clean up log once `scope="local"` (#40540)
Wentao Ye committed 7 months ago
[Bugfix][Parser] Fix Mistral pre-v11 tool parser failing on trailing model output (#40531)
Doug Smith committed 7 months ago
FlexAttention non-causal support (#40394)
Fynn Schmitt-Ulms committed 7 months ago
Fix test_startup.py for torch 2.12 (#40636)
Angela Yi committed 7 months ago