[Core] Persist and reuse the memory-profiling result across boots (opt-in) (#47388)

Nils Matteson committed 4 months ago

[ROCm][Bugfix] Key sparse-MLA persistent metadata on per-request context lengths (#47766)

Rohan Potdar committed 4 months ago

[ROCm][CI] Minimize comment in RocmAttention q_scale check (#47731)

stefankoncarevic committed 4 months ago

[CI/Build] Accept ready-run-all-tests label in pre-commit gate (#47897)

Ameen Patel committed 4 months ago

[Bugfix] Patch Hopper MXFP4 OOB scales reads leading to NaN (#47910)

yzong-rh committed 4 months ago

fix(security): bound completion prompt list to prevent unbounded engine fan-out (#47845)

Juan Pérez de Algaba committed 4 months ago

[BugFix] Fix ModelOpt quantization inference for fused siblings (#47445)

Jason Li committed 4 months ago

Add tuned selective_state_update config for AMD Instinct MI355 (#47767)

vanshbhatia-amd committed 4 months ago

[Doc] Surface the --kv-cache-memory suggestion at INFO and document fast-startup knobs (#47374)

Nils Matteson committed 4 months ago

[Bugfix] Pad Mamba page size instead of scaling block_size in unify_kv_cache_spec_page_size (#45207)

Sahil Kadadekar committed 4 months ago

[CI Bug Fix] Temp fix for v3.2 accuracy (#47902)

Wentao Ye committed 4 months ago

[Doc] Fix manylinux tag in installation guide (#47913)

Nick Cao committed 4 months ago

[Bugfix] Allow non-contiguous query in FlashInfer FP8 query quantization (#47908)

Matthew Bonanni committed 4 months ago

[Bugfix] Fix mamba+dflash for MRV2 (#47698)

Benjamin Chislett committed 4 months ago

fix: include topic frame in KV events replay response (#45177)

Rishabh Saini committed 4 months ago

[Bugfix] Exclude kv_cache_memory_bytes from CacheConfig.compute_hash (#47356)

Nils Matteson committed 4 months ago

Upgrade tpu-inference to v0.24.0 (#47835)

Sting Lin committed 4 months ago

[Perf][3/N] Expand Triton kernel warmup coverage, Qwen (#47546)

Roberto L. Castro committed 4 months ago

[Doc] Fix grammatically incorrect error message in gpu_worker and xpu_worker (#36715)

Robin committed 4 months ago

[XPU] Fix topk_sigmoid arg mismatch on XPU (#47858)

liuzhenwei committed 4 months ago