[KV Offloading] Add tiering metric plumbing (#45959)

Srinivas Krovvidi committed 4 months ago

[CPU][Bugfix][Speculative Decoding] Accept USE_FP64_GUMBEL in CPU recovered-tokens sampler (#46069)

hillel.darshan committed 4 months ago

[Frontend] Split ServingRender into renderer and entrypoint. (#44285)

wang.yuqi committed 4 months ago

[ROCm][P/D] Support MoRIIO heterogeneous TP fan-in (#46332)

Tan Pin Siang committed 4 months ago

fix(security): prevent infinite loop in split_audio with NaN audio sa… (#46463)

Juan Pérez de Algaba committed 4 months ago

[ROCm][Perf] DSv3.2: fuse MLA Q concat+fp8-quant in forward_mqa (#43673)

frida-andersson committed 4 months ago

fix gpt_oss pp>1 with ep (#46441)

Qiming Zhang committed 4 months ago

[Rust Frontend] Align Rust allowed_token_ids validation with Python (#46348)

Reid committed 4 months ago

[Rust Frontend] Support echo for token-ID completion prompts (#46219)

Reid committed 4 months ago

Doc: fix missing GLM-5.x in supported models (#46452)

Sunny Yuan committed 4 months ago

[Rust Frontend] Correct `--reasoning-parser` semantics (#46359)

Bugen Zhao committed 4 months ago

[Bugfix][Model Runner V2] Preserve all allowed_token_ids in the logit bias kernel (#46245)

Ting SUN committed 4 months ago

[BugFix] Omit empty tool_calls from OpenAI chat responses (#44105)

jack committed 4 months ago

[Refactor] Responses API parser state into conversation context (#46030)

Chauncey committed 4 months ago

[DeepEP V2] Fill invalid recv_topk_idx with -1 (#46432)

Woosuk Kwon committed 4 months ago

[ROCm][P/D] Fix MoRIIO WRITE mode for mixed KV layouts (#46290)

Tan Pin Siang committed 4 months ago

[Rust Frontend][Perf] Use dedicated runtime for HTTP/request-processing/ZMQ (#46051)

Bugen Zhao committed 4 months ago

[XPU][CI]fix xpu kv cache layout test (#46429)

Kunshang Ji committed 4 months ago

[v1][kvcache] Honor prefix-cache retention interval for Mamba/linear attention (#45845)

Dao007forever committed 4 months ago

[Quant] Enable modelopt_mixed on Turing (SM75) (#45375)

Mike G committed 4 months ago