update marlin M size for EP (#48144)

gnovack committed 4 months ago

Correct model layer aliasing for Bert style models (#43896)

ap9272 committed 4 months ago

[CI] Fix cargo-deny config flag ordering (#48170)

Lucas Wilkinson committed 4 months ago

[ROCm] Revert Part of `[ROCm] Fix pooling startup workspace lock` #47912 (#48154)

Micah Williamson committed 4 months ago

[CI] Increase extract hidden states TP2 timeout (#48161)

Lucas Wilkinson committed 4 months ago

[ROCm] Synchronize sparse MLA metadata before graph replay (#47404)

ZihaoMu committed 4 months ago

[KV Connector][Mooncake] Apply SWA lookup mask before hashing/key build (#47317)

Zhewen Li committed 4 months ago

[ROCM][DSV32][Perf][MTP] Enable UNIFORM_BATCH CG mode in rocm_aiter_mla_sparse (#45149)

Teemu Virolainen committed 4 months ago

[Bugfix][MRV2] Reset num_accepted_tokens on add_request in all modes (#48132)

Nick Hill committed 4 months ago

[KVConnector] MultiConnector: give every sub-connector the request's real blocks in `update_state_after_alloc` (#46865)

weishu committed 4 months ago

[Bugfix] Preserve tensor causal metadata for grouped attention (#48135)

Lucas Wilkinson committed 4 months ago

[ROCm][CI] Set all timeout_in_minutes to 180 (#48146)

Charlie Fu committed 4 months ago

[Bugfix] Guard CUDA-only rms_norm_per_block_quant in FUSED_OPS for non-CUDA builds (#47296)

Tsvika Shapira committed 4 months ago

Pin PyNvVideoCodec to tested 2.0.4 wheel (#48056)

Brandon Pelfrey committed 4 months ago

[CI] Annotate built Docker image tags on the Buildkite build page (#48101)

Kevin H. Luu committed 4 months ago

[Bugfix][Model Runner V2] Order uniform decodes first so spec decodes aren't misclassified as prefills (#47381)

Woosuk Kwon committed 4 months ago

Migrate Olmo and Olmo2 to the Transformers modeling backend (#48100)

Harry Mellor committed 4 months ago

Remove PersimmonForCausalLM and FuyuForCausalLM model architectures (#48096)

Tiezhen WANG committed 4 months ago

[P/D][Bugfix] Fix PD async KV load lookahead handling for MTP spec decode (#46694)

Chauncey committed 4 months ago

Sanitize server file paths from validation error responses (#46415)

Muhammad Fawaz committed 4 months ago