@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 5, 12:15 UTC

Code Commits 22,390 Issues Pulls Cloud Agent CICD
main
main

[Bug] Fix mistral version dependency (#39086)

Wentao Ye committed 7 months ago

419e73cdfa

[MoE Refactor][Test] FusedMoE layer test (#24675)

bnellnm committed 7 months ago

f01482408c

[NIXL][Mamba][3/N] Heterogeneous TP: 3-read conv state transfer (#37635)

zhanqiuhu committed 7 months ago

bfdc0a3a99

[MoE Refactor] Split of DefaultMoERunner class (#35326)

bnellnm committed 7 months ago

93bada494f

[Core] Re-enable Inductor pre-grad passes in standalone compile (torch>=2.12) (#38944)

Frederik Gossen committed 7 months ago

608914de30

[Refactor] Remove unused dead code (#38842)

Wentao Ye committed 7 months ago

4ae218c122

[Models][GDN] Remove GPU/CPU syncs in `GDNAttentionMetadata.build` during speculative decoding (#38047)

Lukas Geiger committed 7 months ago

f40d9879f2

[Gemma4] Enable Fast Prefill Optimization (#38879)

Lucas Wilkinson committed 7 months ago

47e605092b

[Feat][Core] safely abort requests when FSM fails to advance (#38663)

Walter Beller-Morales committed 7 months ago

e69a265135

[Mistral Grammar] Support Grammar Factory (#38150)

Julien Denize committed 7 months ago

fef56c1855

[Model] Add support for BharatGen's Param2MoE model (#38000)

bhargav-patel-29 committed 7 months ago

c5e3454e5a

MiniMax-M2: add Eagle3 speculative decoding support (#37512)

liuchenbing2026 committed 7 months ago

f6983f01de

[ROCm][Quantization] Add asymmetric INT8 quantization support to TritonInt8ScaledMMLinearKernel (#38501)

Andreas Karatzas committed 7 months ago

780ba37458

[ROCm][CI] Run Kernels Core Operation Test On MI325 and mitigate flakiness (#38184)

Micah Williamson committed 7 months ago

9570654c6d

nano_nemotron_vl: fix tensor device mismatch exception when video profiling (#39029)

Netanel Haber committed 7 months ago

d56e952239

[ci] Switch some CI jobs to H200 MIG slices (#38956)

Kevin H. Luu committed 7 months ago

56de443db1

[Bug] Fix Import paths for `encoder_cudagraph` modules (#38997)

Greg Pereira committed 7 months ago

4dd49b06f8

[Bugfix] Fix invalid JSON in Gemma 4 streaming tool calls by stripping partial delimiters (#38992)

Greg Pereira committed 7 months ago

f53fa26e05

[Perf] Change Trtllm fp8 MoE to use Shuffled Weights and BlockMajorK Layout (#38993)

Wei Zhao committed 7 months ago

1af6f78ae5

[Bugfix][MoE] Fix 6-8% decode regression: prefer multi-stream shared expert overlap (#38990)

Martin Vit committed 7 months ago

228023b3a5
← Newer Page 343 of 1120 Older →