A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 03:30 UTC
main
Enable ZP Support for Machete (#20268)
czhu-cohere committed over 1 year ago
[Misc] add xgrammar for arm64 (#18359)
Prashant Gupta committed over 1 year ago
Fix `numel()` downcast in vllm/csrc/moe/moe_align_sum_kernels.cu +2 (#17082)
Richard Barnes committed over 1 year ago
[V1] Only print cudagraph tqdm on rank 0 with `is_global_first_rank` (#19516)
Michael Goin committed over 1 year ago
Alex Kogan committed over 1 year ago
[Doc] add config and troubleshooting guide for NCCL & GPUDirect RDMA (#15897)
Ernest Wong committed over 1 year ago
[CI][Intel Gaudi][vllm-Plugin]Add CI for hpu-plugin-v1-test (#20196)
Chendi.Xue committed over 1 year ago
[Example] add one-click runnable example for P2P NCCL XpYd (#20246)
Kuntai Du committed over 1 year ago
[Misc] Fix spec decode example (#20296)
Woosuk Kwon committed over 1 year ago
[Bugfix] Fix include prompt in stream response when echo=true (#15233)
fyuan1316 committed over 1 year ago
[CLI] Improve CLI arg parsing for `-O`/`--compilation-config` (#20156)
Luka Govedič committed over 1 year ago
[Bugfix][V1][P/D]Fix the issue of occasional garbled output for P2pNcclConnector (#20263)
Zhonghua Deng committed over 1 year ago
[Refactor] Remove useless pdb comment (#20266)
Wentao Ye committed over 1 year ago
[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)
Kyle Sayers committed over 1 year ago
[Unit Test] Add unit test for deep gemm (#20090)
Wentao Ye committed over 1 year ago
[Optimization] Use Shared `CachedRequestData` Instance Across All Requests (#20232)
Woosuk Kwon committed over 1 year ago
[Spec Decode] Clean up spec decode example (#20240)
Woosuk Kwon committed over 1 year ago
[Spec Decode] Refactor spec decoding into a separate function (#20238)
Woosuk Kwon committed over 1 year ago
[Bugfix] fix quark ptpc (#20251)
li haoyang committed over 1 year ago
[Docs] Fix 1-2-3 list in v1/prefix_caching.md (#20243)
Michael Yao committed over 1 year ago