A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[Benchmarks] Throw usage error when using dataset-name random and dataset-path together (#24819)
Ye (Charlotte) Qi committed about 1 year ago
fix type of sampling rate for encode_base64 (#24826)
co63oc committed about 1 year ago
[Perf] Fix DeepGEMM Contiguous Layout Issue, 5.5% Throughput Improvement (#24783)
Wentao Ye committed about 1 year ago
[Misc] Improve `s3_utils` type hints with `BaseClient` (#24825)
Hyogeun Oh (오효근) committed about 1 year ago
[Multi Modal][Performance] Fused Q,K's apply_rope into one (#24511)
Wenlong Wang committed about 1 year ago
[Chore] Minor simplification for non-PP path (#24810)
Woosuk Kwon committed about 1 year ago
[Minor] Simplify duplicative device check for cuda (#24793)
Victor Ziliang Peng committed about 1 year ago
Remove redundant assignment in xfer_buffers, This is a little fix (#24732)
TaoYu Chen committed about 1 year ago
[CI][Spec Decode] Adjust threshold for flaky ngram spec decoding test again (#24771)
Wenlong Wang committed about 1 year ago
[Doc]: fix typos in various files (#24798)
Didier Durand committed about 1 year ago
[Misc] Correct an outdated comment. (#24765)
Russell Bryant committed about 1 year ago
[CI Failure] Fix test_flashinfer_cutlass_mxfp4_mxfp8_fused_moe (#24750)
Michael Goin committed about 1 year ago
[Core][Multimodal] Cache `supports_kw` (#24773)
Lukas Geiger committed about 1 year ago
[Kernels][DP/EP] Optimize Silu Kernel for R1 (#24054)
Elvir Crnčević committed about 1 year ago
[Perf] Use NVIDIA hardware-accelerated instruction for float to fp8_e4m3 quantization (#24757)
elvischenv committed about 1 year ago
[Doc]: Remove 404 hyperlinks (#24785)
Rakesh Asapanna committed about 1 year ago
[Docs] Remove Neuron install doc as backend no longer exists (#24396)
Harry Mellor committed about 1 year ago
[Docs] Fix warnings in mkdocs build (continued) (#24791)
Hyogeun Oh (오효근) committed about 1 year ago
[Chore] Remove unused batched RoPE op & kernel (#24789)
Woosuk Kwon committed about 1 year ago
[Bugfix] Fix GPUModelRunner has no attribute lora_manager (#24762)
Jee Jee Li committed about 1 year ago