A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 9, 10:00 UTC
main
Bugfix: Pass router logits dtype in nemotron shared experts (#32669)
amirkl94 committed 9 months ago
Make `mypy` opt-out instead of opt-in (#33205)
Harry Mellor committed 9 months ago
[Misc] Remove missed `pad_for_cudagraph` (#33283)
Lucas Wilkinson committed 9 months ago
[Doc] Update outdated link to Ray documentation (#32660)
graftim committed 9 months ago
Adding optional speculator tests for larger models (#32943)
shanjiaz committed 9 months ago
[PluggableLayer][2/N] Apply PluggableLayer to linear layers (#33152)
whx committed 9 months ago
support returning tokenids in responses api (#33212)
cmunley1 committed 9 months ago
[BugFix] Fix EPLB fail for MoeFP4 model with Marlin backend (#33262)
Ilya Markov committed 9 months ago
[Doc]: fixing multiple typos in diverse files (#33256)
Didier Durand committed 9 months ago
[Bugfix] Fix Qwen3-VL-Reranker load. (#33298)
wang.yuqi committed 9 months ago
[CI/Build][BugFix] fix cuda/compat loading order issue in docker build (#33116)
Pengchao Wang committed 9 months ago
[Release] [ROCm] Remove old build step (#33316)
TJian committed 9 months ago
[Misc][Build] Lazy load cv2 in nemotron_parse.py (#33189)
Kiersten Stokes committed 9 months ago
[Release] [CI] Optim release pipeline (#33156)
TJian committed 9 months ago
Fix tool call indexing double-counting (#33141)
wangln19 committed 9 months ago
[Refactor] Define MM data parser in processing info instead of processor itself (#33260)
Cyrus Leung committed 9 months ago
[ez] Delete more torch version checks <= 2.8 (#33288)
Angela Yi committed 9 months ago
[Misc] Add orozery to CODEOWNERS (core, kv_transfer, kv_offload) (#33227)
Or Ozeri committed 9 months ago
[Bugfix] Register fp8 cutlass_group_gemm as supported for only SM90+SM100 (#33285)
Michael Goin committed 9 months ago
[UX] Remove noisy CT UnquantizedLinearMethod warn (#33273)
Michael Goin committed 9 months ago