[BUGFIX] deepseek-v2-lite failed due to fused_qkv_a_proj name update (#21414)

Chendi.Xue committed over 1 year ago

[BugFix] Update python to python3 calls for image; fix prefix & input calculations. (#21391)

ericehanley committed over 1 year ago

Simplify weight loading in Transformers backend (#21382)

Harry Mellor committed over 1 year ago

[Bugfix][ROCm][Build] Fix build regression on ROCm (#21393)

Gregory Shtrasberg committed over 1 year ago

[CI/Build] Fix model executor tests (#21387)

Cyrus Leung committed over 1 year ago

[BugFix] Fix ray import error mem cleanup bug (#21381)

Joe Runde committed over 1 year ago

[Misc] Copy HF_TOKEN env var to Ray workers (#21406)

Rui Qiao committed over 1 year ago

[Model] Add Qwen3CoderToolParser (#21396)

Yiheng Xu committed over 1 year ago

Fix Flashinfer Allreduce+Norm enable disable calculation based on `fi_allreduce_fusion_max_token_num` (#21325)

Xin Li committed over 1 year ago

[CI/Build] Fix test failure due to updated model repo (#21375)

Cyrus Leung committed over 1 year ago

[Bugfix] Decode Tokenized IDs to Strings for `hf_processor` in `llm.chat()` with `model_impl=transformers` (#21353)

Aritra Roy Gosthipaty committed over 1 year ago

Add tokenization_kwargs to encode for embedding model truncation (#21033)

Wang Yijun committed over 1 year ago

Revert "[Refactor] Fix Compile Warning #1444-D (#21208)" (#21384)

Wentao Ye committed over 1 year ago

[feat] Enable mm caching for transformers backend (#21358)

Raushan Turganbay committed over 1 year ago

Adds parallel model weight loading for runai_streamer (#21330)

Benjamin Bartels committed over 1 year ago

[Perf] Cuda Kernel for Per Token Group Quant (#21083)

Wentao Ye committed over 1 year ago

[feat]: add SM100 support for cutlass FP8 groupGEMM (#20447)

Duncan Moss committed over 1 year ago

[perf] Add fused MLA QKV + strided layernorm (#21116)

Mickaël Seznec committed over 1 year ago

[Misc] unify variable for LLM instance v2 (#21356)

Ning Xie committed over 1 year ago

[Core] Introduce popleft_n and append_n in FreeKVCacheBlockQueue to further optimize block_pool (#21222)

Jialin Ouyang committed over 1 year ago