[Bugfix] Fix KV Scale loading for MLA Models (#35430)

Pavani Majety committed 8 months ago

[ROCm][Quantization] GPT OSS Upstream MoE wmxfp4_afp8 with static scales (#30357)

Aleksandr Malyshev committed 8 months ago

[Kernel][perf] optimize NCCL symm_mem vs custom_AR selection thresholds (#33839)

pkousha committed 8 months ago

[WideEP] Remove pplx all2all backend (#33724)

Tyler Michael Smith committed 8 months ago

[Bugfix] Fix MessageQueue connect_ip for cross-node data parallelism (#35429)

Lucia Fang committed 8 months ago

add mixed precision support for modelopt (#35047)

sychen52 committed 8 months ago

Nemotron: use per-layer config in NemotronHMLPDecoderLayer for heterogeneous models (#35396)

danielafrimi committed 8 months ago

[Performance] Extract KV cache update op from flashinfer forward (#35422)

ElizaWszola committed 8 months ago

fix(reasoning): Qwen3ReasoningParser returns truncated output as reasoning (#35230)

不做了睡大觉 committed 8 months ago

[Model Runner V2] Prepare attn metadata in ModelState [2/N] (#35383)

Woosuk Kwon committed 8 months ago

[Model Runner V2] Add model states [1/N] (#35350)

Woosuk Kwon committed 8 months ago

[Model Runner V2] Fix error-handling (#35063)

Nick Hill committed 8 months ago

[Bugfix] Remove erroneous lower bound on LoRA vocab size constraint (#35354)

Lucas Wilkinson committed 8 months ago

[BugFix] Align fused MoE-LoRA kernel config with actual weight shapes (#34396)

Runkai Tao committed 8 months ago

[Refactor] Remove dead code for attention benchmark script (#35418)

Wentao Ye committed 8 months ago

[Core] Support `min_tokens` with speculative decoding (#32642)

Yiliu Dong committed 8 months ago

[Perf] Optimize maxsim scores computation for pooling models, 13.9% E2E throughput improvement (#35330)

Wentao Ye committed 8 months ago

Add GlmOcrConfig for GLM-OCR model type recognition (#34982)

hujia177 committed 8 months ago

[BugFix][kv_offload]: Fix kernel block size detection (#35125)

Or Ozeri committed 8 months ago

[ROCm] Update the torch version in rocm_build.txt to use the official 2.10 release (#34387)

Sage Moore committed 8 months ago