[Misc][Nixl] Unify `_logical_to_remote_kernel_block_ids` (#48717)

Nicolò Lucchesi committed 3 months ago

[KV Offload] Define clean backend configuration boundary (#48150)

Chang Guo committed 3 months ago

[Bugfix][Rust Frontend] Limit chat top_logprobs in responses (#48134)

Reid committed 3 months ago

Vectorize prep xfer list creation (#48209)

Ilia Yastrebov committed 3 months ago

[Rust Frontend] Add Seed-OSS tool parser (#47741)

Chao-Ju Chen committed 3 months ago

[Rust Frontend] Wait for mock engine endpoints before ZMQ connect (#47965)

Reid committed 3 months ago

[Bugfix][Rust Frontend] Detokenizer: avoid leaking prompt on zero-generated-token completions (#47707)

JinYan Su committed 3 months ago

BF16x3 router GEMM (#47973)

Thien Tran committed 3 months ago

[Rust Frontend] Fix mock engine test shutdown race (#48738)

Reid committed 3 months ago

[Bugfix] Fix offloading set_ overflow for packed non-uniform KV caches (#48530)

Elvir Crnčević committed 3 months ago

[Model] Add Inkling model support [1/N] (#48799)

Woosuk Kwon committed 3 months ago

[Spec Decode] Add kv_cache_dtype to speculative_config to control separately from target (#48787)

Michael Goin committed 3 months ago

[ROCm][CI] Set "highest" matmul precision for reference hf_runner in `test_bert_for_masked_lm` (#48784)

Micah Williamson committed 3 months ago

fix(lora): validate LoRA rank is positive in PEFTHelper (#48437)

ErenAta16 committed 3 months ago

[ROCm][CI] Fix cuda graph mem profile issue (#48764)

Charlie Fu committed 3 months ago

Bump flashinfer version to 0.6.14 (#47669)

Ameen Patel committed 3 months ago

[BugFix] Don't apply weight in batch-invariant RMSNorm when has_weight=False (#48741)

Asaf Gardin committed 3 months ago

[Bugfix][Spec Decode] Fix eagle3 first-layer qkv_proj prefix for quantized drafts (#48068)

qizixi committed 3 months ago

[Docs] fix error key name (#48802)

rongfu.leng committed 3 months ago

[CI] Fix flaky lora test (#47375)

qli88 committed 3 months ago