A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[CI/Build] Increase pooling tolerance to pass CI (#22844)
Cyrus Leung committed over 1 year ago
[Core] [N-gram SD Optimization][1/n] Propose tokens with a single KMP (#22437)
Jialin Ouyang committed over 1 year ago
Move checklist in PR template (#22852)
Luka Govedič committed over 1 year ago
[ROCm][Bugfix] Fix compilation error in topk softmax fused kernel (#22819)
kliuae committed over 1 year ago
[CI/Build] Skip gpt_big model test because of broken HF model (#22848)
Isotr0py committed over 1 year ago
[CI/Build] Fix param mismatch in `test_eagle_correctness` (#22847)
Cyrus Leung committed over 1 year ago
[CI] Fix `tests/v1/e2e/test_kv_sharing_fast_prefill.py` import on test (#22815)
Nicolò Lucchesi committed over 1 year ago
[CI/Build] Update VLM common tests (#22841)
Cyrus Leung committed over 1 year ago
[Bugfix] Fix MiniCPMV Image input inference failed (#22813)
HWH committed over 1 year ago
[Misc] clear and separate error messages for input too long and input + max-tokens too long (#22803)
Roger Wang committed over 1 year ago
[Core] Use individual MM items in P0/P1 cache and model runner (#22570)
Cyrus Leung committed over 1 year ago
[Frontend] Multithreaded async multimodal load_bytes (#22710)
milesial committed over 1 year ago
Fix GGUF loader for Qwen3 MoE. (#22785)
Gh0u1L5 committed over 1 year ago
Remove unnecessary CUDA sync of qwen image and video preprocess (#22792)
Yuanyuan Chen committed over 1 year ago
[Bugfix][mamba] Fix type annotation of Mamba2Metadata (#22787)
Chen Zhang committed over 1 year ago
Nicolò Lucchesi committed over 1 year ago
[FEATURE] support custom vllm tuned config path for fused moe triton kernels (#22791)
Chi Zhang committed over 1 year ago
[Frontend] Add chunked processing to handle long inputs in embedding models (#22280)
Kdump committed over 1 year ago
[Platform] Custom ops support for FusedMoe (#22509)
wangxiyuan committed over 1 year ago
[V1] Add tree drafting tests for eagle spec decoding (#22705)
Giancarlo Delfin committed over 1 year ago