A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 9, 16:00 UTC
main
feat(benchmark): add encoder forward pass benchmarking to mm-processor (#31655)
Reagan Lee committed 10 months ago
[Doc] Ignore typo check on doc (#32999)
Roger Wang committed 10 months ago
[Models] Add `SharedFusedMoE` support to Qwen3MoE (#32082)
Isotr0py committed 10 months ago
[docs] Update governance process links (#32995)
Roy Wang committed 10 months ago
[Tests] Standardize RNG seed utility across test files (#32982)
7. Sun committed 10 months ago
[Tests] Clarify pytest skip reasons with actionable context (#32981)
7. Sun committed 10 months ago
[Perf] Cache xpu_get_mem_info() result to avoid duplicate calls (#32983)
7. Sun committed 10 months ago
[Dev UX] Add auto-detection for VLLM_PRECOMPILED_WHEEL_VARIANT during install (#32948)
Michael Goin committed 10 months ago
Auth_token added in documentation as it is required (#32988)
ruizcrp committed 10 months ago
[ROCm][ViT] Enable Flash Attention Triton backend on RDNA3/RDNA4 (#32944)
monajafi-amd committed 10 months ago
[Bugfix] Fix FusedMoE LoRA kernel offs_token out of bound value (#32279)
Xin Yang committed 10 months ago
[Core][Bugfix] allow graceful worker termination (#32965)
Joe Runde committed 10 months ago
[Performance] Split FlashAttn attention and cache update (#25954)
ElizaWszola committed 10 months ago
[fix] add VLLM_OBJECT_STORAGE_SHM_BUFFER_NAME to compile factors (#32912)
dolpm committed 10 months ago
[CI] fix version comparsion and exclusion patterns in upload-release-wheels.sh (#32971)
Shengqi Chen committed 10 months ago
[Bugfix] Fix missing is_layer_skipped check for FusedMoE in AWQConfig (#32935)
joninco committed 10 months ago
[Refactor] Clean up unused variables & func (#32692)
Wentao Ye committed 10 months ago
[Refactor] Rename `gptq_marlin` to `marlin` to match MoE (#32952)
Michael Goin committed 10 months ago
rasmith committed 10 months ago
[Bug] Fix benchmark script `moe_permute_unpermute` (#32949)
Wentao Ye committed 10 months ago