[Core] Add dynamo_timed tracing for print_readable (#40834)

Frederik Gossen committed 2 months ago

(security) fix: enforce decoder prompt-length validation for skip-che… (#46588)

Juan Pérez de Algaba committed 2 months ago

[Bugfix] Return HTTP 500 for non-streaming generate errors (#49195)

Wei-Cheng (Wayne) Chiu committed 2 months ago

[Bugfix] Include device index in compile cache paths (#38962)

Neil Schemenauer committed 2 months ago

[ROCm][Perf] Kimi-K3 Fused kernels for KDA prefill (#52606)

kliuae committed 2 months ago

Remove native Hunyuan V1 and VL implementations (#53272)

Tiezhen WANG committed 2 months ago

[Bugfix] Fix MTP draft model using local cache path instead of S3 URL with runai_streamer (#42079)

SoluMilken committed 2 months ago

[DCP] Default query replication for GLM sparse attention (#50382)

Lucas Wilkinson committed 2 months ago

[ROCm] Cpu offload for ROCm 7.13+ to align the hipMemcpyBatchAsync params and perf in 7.14x (#43018)

Hongxia Yang committed 2 months ago

[Bugfix] Fix HYV3 shared_mlp prefix for compressed-tensors ignore matching (#48682)

DCo committed 2 months ago

[Bugfix] test_batch_inference_correctness now uses batch invariance (#48040)

Morrison Turnansky committed 2 months ago

[Bugfix] Fix six quantization exception messages split across positional args (#50479)

ErenAta16 committed 2 months ago

using existing uvicorn configuration for dp supervisor (#52473)

Greg Pereira committed 2 months ago

[Frontend] Prevent Kimi K3 reserved markers in response text (#52889)

Bugen Zhao committed 2 months ago

[Docs] document cache salting for prefix cache timing side-channel mitigation (#39082)

Russell Bryant committed 2 months ago

[Bugfix][Spec Decode]Preserve user --speculative-config overrides for speculators-format models (#42376)

Lee Yongjun committed 2 months ago

[Bugfix][Attention] Normalize FlashInfer prefill LSE before merging (#52796)

yimdev committed 2 months ago

[Kernel] Add b12x FP4 MoE backend (#52018)

Luke Alonso committed 2 months ago

[CI][Bugfix] Use a prompt that survives offload-resume rounding in mamba offload test (#53146)

Frederik Gossen committed 2 months ago

[Bugfix] Deterministic MoE combine (reduce_scatterv) under VLLM_BATCH_INVARIANT (#45683)

shijuzhao committed 2 months ago