A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 23:45 UTC
main
[Attention] Make CutlassMLA the default backend for SM100 (blackwell) (#21626)
Alexander Matveev committed over 1 year ago
[Misc] Refactor vllm config str (#21666)
Ning Xie committed over 1 year ago
Fix CUDA permute/unpermute for use with DeepGemm Moe (#17934)
Caleb_Du committed over 1 year ago
[Refactor] Refactor MOE NVFP4 Code Base: ModelOpt + Compressed Tensor (#21631)
Wentao Ye committed over 1 year ago
[VLM] Add video support for Intern-S1 (#21671)
Isotr0py committed over 1 year ago
Migrate Florence2ImagePixelInputs to TensorSchema (#21663)
Benji Beck committed over 1 year ago
[Misc] add default value for file pattern arg (#21659)
Ning Xie committed over 1 year ago
Refactor: Remove numpy dependency from LoggingStatLogger (#20529)
ZiTian.Zhao committed over 1 year ago
[CI/Build][Doc] Clean up more docs that point to old bench scripts (#21667)
Ye (Charlotte) Qi committed over 1 year ago
Handle non-serializable objects in vllm bench (#21665)
Huy Do committed over 1 year ago
[VLM] Support HF format Phi-4-MM model (#17121)
Isotr0py committed over 1 year ago
Migrate ChameleonImagePixelInputs to TensorSchema (#21657)
Benji Beck committed over 1 year ago
Migrate FuyuImagePatchInputs to TensorSchema (#21662)
Benji Beck committed over 1 year ago
Migrate DeepseekVL2ImageInputs to TensorSchema (#21658)
Benji Beck committed over 1 year ago
Migrate Blip2ImagePixelInputs and Blip2ImageEmbeddingInputs to TensorSchema (#21656)
Benji Beck committed over 1 year ago
support `torch.compile` for bailing moe (#21664)
Jinzhen Lin committed over 1 year ago
Remove xformers requirement for Mistral-format Pixtral and Mistral3 (#21154)
Wenchen Lo committed over 1 year ago
[NVIDIA] Explicitly disable shuffled weights for flashinfer blockscale moe fp8 kernels (#21411)
Kaixi Hou committed over 1 year ago
[CI/Build][Doc] Move existing benchmark scripts in CI/document/example to vllm bench CLI (#21355)
Ye (Charlotte) Qi committed over 1 year ago
Yeju Zhou committed over 1 year ago