A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 07:15 UTC
main
[P/D][NixlConnector] Enable FlashInfer backend (#19090)
Nicolò Lucchesi committed over 1 year ago
[MISC][Bugfix] Use less CPU when message queue has been empty for some time (#16226)
Povilas Kanapickas committed over 1 year ago
[Hardware][NVIDIA] FP4 MoE kernel optimization (#19110)
Chiyue Wei committed over 1 year ago
Add H20-3e fused MoE kernel tuning configs for DeepSeek-R1/V3 (#19205)
Xu Wenqing committed over 1 year ago
[mistral_common] Add v11 tokenizer (#19193)
Patrick von Platen committed over 1 year ago
Guillaume Calmettes committed over 1 year ago
[Frontend] improve vllm run-batch --help display (#19187)
Reid committed over 1 year ago
[Misc] Do not override NCCL_CUMEM_ENABLE if set explicitly (#19105)
22quinn committed over 1 year ago
[Misc] Remove unnecessary fallback to prefill-decode attention (#19138)
vllmellm committed over 1 year ago
[Build] Annotate wheel and container path for release workflow (#19162)
Simon Mo committed over 1 year ago
[Bugfix]: Fix the incompatibility issue with stream when Thinking is disabled (#19135)
Chauncey committed over 1 year ago
[BugFix][Minor] Fix full cuda graph bug when max_num_seqs < 512 (#19171)
Woosuk Kwon committed over 1 year ago
Handle non-serializable objects when dumping benchmark results (#19114)
Huy Do committed over 1 year ago
[Torch Nightly]add missing dependency (#18770)
Yang Wang committed over 1 year ago
[Bugfix] Fix port handling in make_zmq_path (#19117)
Michael Goin committed over 1 year ago
[Bugfix][Nixl] Fix full prefix cache hit bug (#18632)
Robert Shaw committed over 1 year ago
Reid committed over 1 year ago
[P/D] Heterogeneous TP (#18833)
Nicolò Lucchesi committed over 1 year ago
[Misc] refactor: simplify EngineCoreClient.make_async_mp_client in AsyncLLM (#18817)
CYJiang committed over 1 year ago
[Kernel] Integrate batched/masked deepgemm kernel (#19111)
Varun Sundar Rabindranath committed over 1 year ago