[DSv4 Perf] Adaptive topk width, 1.0% E2E throughput improvement (#50004)

Wentao Ye committed 3 months ago

[Core] Fail fast when /dev/shm is too small for the shm ring buffer (#48879)

Andrea Tassi committed 3 months ago

[Test] Regression test for hybrid-Mamba eagle cache-peek in Mooncake connector (#43559) (#48361)

Rishi Puri committed 3 months ago

[Bugfix][CPU] Fall back to torch for unaligned swigluoai on NEON/vec MoE (#49985)

oops-oom committed 3 months ago

Fix MLA padding and grouped topk routing in the Transformers modelling backend (#49982)

Harry Mellor committed 3 months ago

[Attention] Integrate FlashAttention 4 SM100 headdim 256 support (#42669)

Matthew Bonanni committed 3 months ago

[MRV2] Always build attn metadata at capture time (#49364) (#49995)

Nick Hill committed 3 months ago

[Bugfix] Changed speech to text chunk timestamp to cumulative approach (#41131)

TobyJBell committed 3 months ago

[Rust Frontend][gRPC] Add server and model discovery (#49491)

Connor Carpenter committed 3 months ago

[Misc][PD] Nixl cleanup `get_backend_aware_kv_block_len` and `virtually_split_kv_in_blocks` (#49988)

Nicolò Lucchesi committed 3 months ago

[Bugfix] Fix VLLM_ENFORCE_STRICT_TOOL_CALLING mutation in tests (#49846)

yzong-rh committed 3 months ago

[Perf] Hash videos by source bytes (#49607)

Guan-Ming Chiu committed 3 months ago

[MRV2][Performance] Skip no-op FP32 logits materialization (#47711)

Song Zhixin committed 3 months ago

[Bugfix] Restore truncate_prompt_tokens for Jina rerank/score online (#49963)

Umut Polat committed 3 months ago

[Bugfix] Reject pipeline parallelism for DiffusionGemma (#45828)

Guan-Ming Chiu committed 3 months ago

[Perf] Tune LL BF16 Router GEMM (#48774)

Roberto L. Castro committed 3 months ago

[Core] Fix internal LB load-balancing (#49204)

Nick Hill committed 3 months ago

[Model] Enable EVS for Qwen3.5 (#48912)

Rui "Garry" Gao committed 3 months ago

[Perf] Make merge attention context count a runtime argument (#48739)

liminfei-amd committed 3 months ago

[Perf] Skip ll_bf16 router GEMM warmup for non-MoE models (#49659)

neweyes committed 3 months ago