A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 02:15 UTC
main
[cold start] replace VLLM_COMPILE_DEPYF with debug_dump_dir (#20940)
Boyuan Feng committed over 1 year ago
[Doc] Clearer mistral3 and pixtral model support description (#20926)
Isotr0py committed over 1 year ago
[Bugfix] Switch bailout logic for kv-cache-dtype with SM100 Flashinfer (#20934)
Pavani Majety committed over 1 year ago
Enabled BnB NF4 inference on Gaudi (#20172)
Ruheena Suhani Shaik committed over 1 year ago
[CI] Fix flaky `test_streaming_response` test (#20913)
Nicolò Lucchesi committed over 1 year ago
feat: add image zoom to improve image viewing experience (#20763)
Reid committed over 1 year ago
[Docs] Add Kuberay to deployment integrations (#20592)
Ricardo Decal committed over 1 year ago
Use w8a8 quantized matmul Pallas kernel (#19170)
XiongfeiWei committed over 1 year ago
[CI/Build] Split Entrypoints Test into LLM and API Server (#20945)
Michael Goin committed over 1 year ago
[Bugfix] Fix incorrect dispatch for CutlassBlockScaledGroupedGemm and DeepGEMM (#20933)
Michael Goin committed over 1 year ago
[BugFix] VLLM_DISABLE_COMPILE_CACHE=1 should disable all reads and writes from the cache (#20942)
Richard Zou committed over 1 year ago
SM100 Cutlass MLA decode with unrestricted num_heads (< 128) for DeepSeek TP (#20769)
Alexander Matveev committed over 1 year ago
Fall back if flashinfer comm module not found (#20936)
Yong Hoon Shin committed over 1 year ago
[Docs] remove outdated performance benchmark (#20935)
Kuntai Du committed over 1 year ago
Fix overflow indexing in causal_conv1d kernel (#20938)
Thomas Parnell committed over 1 year ago
[Misc] Relax translations tests (#20856)
Nicolò Lucchesi committed over 1 year ago
[Misc] ModularKernel : Perform WeightAndReduce inside TritonExperts & DeepGemmExperts (#20725)
Varun Sundar Rabindranath committed over 1 year ago
Add benchmark dataset for mlperf llama tasks (#20338)
Michael Goin committed over 1 year ago
Change default model to Qwen3-0.6B (#20335)
Tyler Michael Smith committed over 1 year ago
[CI/Build] Add Transformers nightly tests in CI (#20924)
Isotr0py committed over 1 year ago