[TPU][Bugfix] fix the missing apply_model in tpu worker (#25526)

Chengji Yao committed about 1 year ago

[Bugfix] [Frontend] Cleanup gpt-oss non-streaming chat tool calls (#25514)

Ben Browning committed about 1 year ago

[Perf] Fix jit compiles at runtime of fla gated delta rule (#25432)

Corey Lowman committed about 1 year ago

[Spec Decode] Enable FlashInfer Spec Decoding (#25196)

Benjamin Chislett committed about 1 year ago

[KV sharing] Re-land Gemma3n model changes from #22628 (#24357)

Yong Hoon Shin committed about 1 year ago

[fix]: add Arm 4bit fused moe support (#23809)

Nikhil Gupta committed about 1 year ago

[BugFix] Fix MLA assert with CUTLASS MLA (#25478)

Lucas Wilkinson committed about 1 year ago

[Compile] Fix AMD Compile Error (#25518)

Wentao Ye committed about 1 year ago

[Core] Support weight_loader_v2 for `UnquantizedLinearMethod` (#23036)

Kyle Sayers committed about 1 year ago

[Bugfix] Use a separate FlashInfer workspace buffer for trtllm-gen (#25520)

Benjamin Chislett committed about 1 year ago

[V1][Metrics] Add per-request TPOT histogram (#24015)

baxingpiaochong committed about 1 year ago

[Model] Support SeedOss Reason Parser (#24263)

0xNullPath committed about 1 year ago

[BUG] Allows for RunAI Streamer and Torch.compile cache to be used together (#24922)

ahao-anyscale committed about 1 year ago

[Bug] Fix AttributeError: 'FusedMoE' object has no attribute 'w13_weight_scale'. Did you mean: 'w13_weight_scale_inv' (#25519)

Wentao Ye committed about 1 year ago

Add CUTLASS FP8 MOE benchmark scripts and kernel config (#25302)

Chenxi Yang committed about 1 year ago

Improve output when failing json.loads() on structured output test (#25483)

Doug Smith committed about 1 year ago

[BugFix] AssertionError: Do not capture num_reqs > max_num_reqs for uniform batch (#25505)

Lucas Wilkinson committed about 1 year ago

[Bugfix] [B200] cutlass_mla - ensure kv_split == 1 for batch size > 1 (#25509)

Alexander Matveev committed about 1 year ago

[Perf] Increase default max splits for FA3 full cudagraphs (#25495)

Lucas Wilkinson committed about 1 year ago

[ROCm] Small functional changes for gptoss (#25201)

Juan Villamizar committed about 1 year ago