[Misc] Fix PhiMoE expert mapping (#21085)

Jee Jee Li committed over 1 year ago

[V1] [KVConnector] Fix MultiprocExecutor worker output aggregation (#21048)

David Ben-David committed over 1 year ago

[V1][P/D]Enhance Performance and code readability for P2pNcclConnector (#20906)

Zhonghua Deng committed over 1 year ago

[Attention] Refactor attention metadata builder interface (#20466)

Lucas Wilkinson committed over 1 year ago

[Bugfix] Fix Machete zero point issue for GPTQ models on SM90 (#21066)

Michael Goin committed over 1 year ago

[TPU] Start using python 3.12 (#21000)

XiongfeiWei committed over 1 year ago

[Docker] Allow FlashInfer to be built in the ARM CUDA Dockerfile (#21013)

Michael Goin committed over 1 year ago

[Bugfix] weight loading use correct tp_group with patch_tensor_parallel_group (#21024)

Kevin_Xiong committed over 1 year ago

Update PyTorch to `torch==2.7.1` for CUDA (#21011)

Michael Goin committed over 1 year ago

Remove torch_xla.tpu.version() from pallas.py. (#21065)

QiliangCui committed over 1 year ago

Support FP8 Quantization and Inference Run on Intel Gaudi (HPU) using INC (Intel Neural Compressor) (#12010)

Nir David committed over 1 year ago

[Model] Remove model sampler (#21059)

Cyrus Leung committed over 1 year ago

Remove Qwen Omni workaround that's no longer necessary (#21057)

Harry Mellor committed over 1 year ago

[fix] fix qwen image_embeds input (#21049)

Avshalom Manevich committed over 1 year ago

feat - add a new endpoint `get_tokenizer_info` to provide tokenizer/chat-template information (#20575)

Mac Misiura committed over 1 year ago

[Model] Consolidate pooler implementations (#20927)

Cyrus Leung committed over 1 year ago

[Docs] Add intro and fix 1-2-3 list in frameworks/open-webui.md (#19199)

Michael Yao committed over 1 year ago

Fix inadvertently silenced PP tests for `mp`, add DeepSeek V2/V3 model family to PP tests (#20831)

Seiji Eicher committed over 1 year ago

[BugFix] Fix import error on non-blackwell machines (#21020)

Lucas Wilkinson committed over 1 year ago

[TPU] fix kv_cache_update kernel block size choosing logic (#21007)

Chengji Yao committed over 1 year ago