[Bugfix] Respect VLLM_WEIGHT_OFFLOADING_DISABLE_PIN_MEMORY in prefetch offloader (#37699)

Yufeng He committed 7 months ago

[Test] Refactor hard coded device string in test files under compile/quantization/models/model_executor folders (#38901)

wliao2 committed 7 months ago

[Attention Backend] TurboQuant: 2-bit KV cache compression with 4x capacity (#38479)

Vibhav Agarwal committed 7 months ago

[Model Runner V2] Disable piecewise cudagraph mode fallback for eagle draft decodes (#39773)

Giancarlo Delfin committed 7 months ago

[CI] Add PyTorch nightly build and test pipeline (#37226)

Andrey Talman committed 7 months ago

[Bugfix] Disable FlashInfer CUTLASS MoE on SM121 (DGX Spark) (#39825)

Michael Goin committed 7 months ago

[Hybrid] Simplify accepted token counting in spec decode for hybrid models (#38372)

Francesco Fusco committed 7 months ago

[Bugfix][Mooncake] Fix thread-local CUDA context for NVLink transfers in _send_blocks (#39548)

Zhewen Li committed 7 months ago

[Bug] Fix batch invariance nvfp4 support (#39820)

Wentao Ye committed 7 months ago

fix(lmcache): correct store for cached requests while enable prefix cache (#39719)

maobaolong committed 7 months ago

[Reasoning][Frontend] Add model config to adjust_request in reasoning parser (#37848)

rishitdholakia13 committed 7 months ago

[MoE Refactor] Refactor ZeroExpertFusedMoE into new framework (#35549)

bnellnm committed 7 months ago

[MoE] Move GPT OSS Triton kernel experts into fused_moe/experts/ (#39007)

Jackmin801 committed 7 months ago

[Mamba] Flashinfer selective_state_update (#36162)

roikoren755 committed 7 months ago

[CI][KVConnector][Metrics] Update multi KV connector edge case according to prefill stats changes (#39808)

zhanqiuhu committed 7 months ago

[Kernel] Support TRTLLM GEN NVFP4 MoE for non-512-aligned hidden dims via weight padding (#39510)

danielafrimi committed 7 months ago

add warning when FP8 KV cache misses prefill query quantization (#39752)

Albert Cheng committed 7 months ago

[KV Offload] Unified memory layout for offloading workers (#37206)

omerpaz95 committed 7 months ago

[ROCm] Align AiterFlashAttentionImpl attn_type check with backend (#39119)

Andrew Barnes committed 7 months ago

[compile] Nest inductor cache under AOT compile dir (#39718)

Alessandro Sangiorgi committed 7 months ago