A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 22:35 UTC
main
[Docs] Update docker.md with HF_TOKEN, new model, and podman fix (#21856)
Michael Goin committed over 1 year ago
[DOC] Fix path of v1 related figures (#21868)
Chen Zhang committed over 1 year ago
[Docs] Switch to better markdown linting pre-commit hook (#21851)
Harry Mellor committed over 1 year ago
[v1][attention] Support Hybrid Allocator + FlashInfer (#21412)
Chen Zhang committed over 1 year ago
[Bugfix] Correct max tokens for non-contiguous embeds (#21798)
milesial committed over 1 year ago
[ci] mark blackwell test optional for now (#21878)
Simon Mo committed over 1 year ago
[ci] add b200 test placeholder (#21866)
Simon Mo committed over 1 year ago
[BugFix] Fix interleaved sliding window not set for Gemma3n (#21863)
Yong Hoon Shin committed over 1 year ago
Gregory Shtrasberg committed over 1 year ago
docker: docker-aware precompiled wheel support (#21127)
Doug Smith committed over 1 year ago
Add `flashinfer_python` to CUDA wheel requirements (#21389)
Michael Goin committed over 1 year ago
[Doc] update Contributing page's testing section (#18272)
David Xia committed over 1 year ago
[Doc] Add FusedMoE Modular Kernel Documentation (#21623)
Varun Sundar Rabindranath committed over 1 year ago
[Docs] use `uv` in GPU installation docs (#20277)
David Xia committed over 1 year ago
[Bugfix] Fix workspace buffer None issue for Flashinfer TRTLLM Backend (#21525)
elvischenv committed over 1 year ago
[Bugfix]fix mixed bits and visual language model quantization in AutoRound (#21802)
Wenhua Cheng committed over 1 year ago
[TPU] Add an optimization doc on TPU (#21155)
Brittany committed over 1 year ago
[Docs] Merge design docs for a V1 only future (#21832)
Harry Mellor committed over 1 year ago
[Bugfix] VLLM_V1 supports passing other compilation levels (#19340)
Richard Zou committed over 1 year ago
[Doc] Update compatibility matrix for pooling and multimodal models (#21831)
Cyrus Leung committed over 1 year ago