[Frontend] Reduce mixin usage in serving pooling (#33101)

Cyrus Leung committed 10 months ago

[Perf] avoid duplicate mem_get_info() call in get_current_memory_usage (#33064)

Paco Xu committed 10 months ago

[DOC]: Add warning about max_num_batched_tokens and max_model_len when chunked prefill is disabled (#33109)

Vincent Gimenes committed 10 months ago

Fix IndexError with encoder-decoder models when using Custom Paged Attention (#33112)

Strahinja Stamenkovic committed 10 months ago

fix: preserve native tool call ID in multi-turn tool calling (#32768)

wangln19 committed 10 months ago

[MoE Refactor] Integrate Naive Prepare Finalize into MK (#32567)

Robert Shaw committed 10 months ago

[Model Runner V2] Remove UvaBufferPool for cpu->gpu copy (#33055)

Woosuk Kwon committed 10 months ago

[Bugfix][TPU] Return a Default fp8 MoE Backend (#32908)

XiongfeiWei committed 10 months ago

[Bugfix][MXFP4] Call `trtllm_fp4_block_scale_moe` with kwargs (#33104)

Pengchao Wang committed 10 months ago

[fix] CPUDNNLGEMMHandler pointer baked into inductor artifact (#32913)

dolpm committed 10 months ago

[Logging] add `--disable-access-log-for-endpoints` CLI option (#30011)

Jared Wen committed 10 months ago

[Refactor] Remove unused `_moe_permute` function (#33108)

Wentao Ye committed 10 months ago

[ci] Sync test areas with test-pipeline.yaml and enable new pipeline generator (#33080)

Kevin H. Luu committed 10 months ago

[Bugfix] Fix Dtypes for Pynccl Wrapper (#33030)

Robert Shaw committed 10 months ago

[Model] Bump transformers version for test registry (#33100)

Cyrus Leung committed 10 months ago

[Bugfix] Fix Voxtral streaming slot_mapping (#33073)

Nicolò Lucchesi committed 10 months ago

[FIX] Always support TP > 4 for FP4 Gemm (#31099)

danielafrimi committed 10 months ago

Remove unused logic in `models/mistral.py` (#33095)

Andy Lo committed 10 months ago

[CI] Fix AssertionError: MCP tool call not found in output_messages (#33093)

Chauncey committed 10 months ago

[ROCm][Bugfix] Fix ptpc scale load issue for fused shared expert path in deepseek mtp (#33018)

Pleaplusone committed 10 months ago