[Spec Decode] Add Qwen3 architecture support for EAGLE3 (#43132)

Benjamin Chislett committed 4 months ago

[Kernel][Bugfix] Fix INT8 per-token-head KV cache rounding in Triton reshape-and-cache (#45361)

ZedongLiu committed 4 months ago

[Multimodal] Add Qwen2-VL/Qwen2.5-VL processor-mapped video loader (#45555)

Ranran committed 4 months ago

[KV Offloading] Add labeled metrics support (#45957)

Srinivas Krovvidi committed 4 months ago

[Model]Fix MiniMaxM2ForCausalLM perf regression (#45935)

Jee Jee Li committed 4 months ago

[Spec Decode] Support mixed KV page sizes for DFlash (#45181)

Alex Steiner committed 4 months ago

[Disagg] return routed_experts on streaming generate responses (#44638)

aoshen02 committed 4 months ago

[ROCm][P/D] Support MiniMax-M3 mixed KV layouts in MoRIIO READ mode (#46039)

junkang1991 committed 4 months ago

[Bugfix] Defer offload reads while transfers are pending (#46231)

Palaiologos1453 committed 4 months ago

[CI][test] Replace InternVL2-1B with InternVL3-1B in test_pipeline_parallel.py (#46241)

xiaolinchen committed 4 months ago

Fix memory pointer overflow in Mamba state buffers (#44665)

Shifani Rajabose committed 4 months ago

[Bugfix][Model Runner V2] Fix min_tokens off-by-one in the V2 GPU sampler (#46243)

Ting SUN committed 4 months ago

[Bugfix] Fix NVFP4/OCP MX MoE emulation (#46254)

Matt committed 4 months ago

[Core] Ensure memory is pinned prior to async h2d copy (#45424)

Nick Hill committed 4 months ago

[Misc] Fix stale doc URL and docstring module path (#35530)

Umut Polat committed 4 months ago

[Render] Add reasoning/tool parsing to /derender + fix byte-fallback FFFD (#45919)

aoshen02 committed 4 months ago

[Perf] Skip/shrink all_token_ids copy in scheduler for non-async and V2 runner (#45840)

aman committed 4 months ago

[SimpleCPUOffloadConnector] PCP + DCP support (#39831)

Jonathan Chen committed 4 months ago

[Perf][KVConnector][Mooncake] Compact chunk-hash keys and zero-copy lookup wire format (#45969)

Yifan Qiao committed 4 months ago

[KV-Offloading] : Expose CPU cache usage metric (#45737)

Varun Sundar Rabindranath committed 4 months ago