[BugFix] Don't cancel asyncio tasks directly from destructors (#22476)

Nick Hill committed over 1 year ago

[Misc] fix openai version (#22485)

rongfu.leng committed over 1 year ago

[Misc] Begin deprecation of `get_tensor_model_*_group` (#22494)

Cyrus Leung committed over 1 year ago

[CI/Build] Fix multimodal tests (#22491)

Cyrus Leung committed over 1 year ago

[bench] Fix benchmark/serve.py to ignore unavailable results (#22382)

lkchen committed over 1 year ago

[Doc] Sleep mode documentation (#22310)

iAmir97 committed over 1 year ago

[bugfix] Fix Llama3/4 issues caused by FlashInfer 0.2.10 (#22426)

Po-Han Huang (NVIDIA) committed over 1 year ago

Fix pre-commit (#22487)

Cyrus Leung committed over 1 year ago

Optimize MiniCPMO mask creation with vectorized implementation (#22464)

ZiTian Zhao committed over 1 year ago

not tie_word_embeddings for glm-4.5 and glm-4.5v (#22460)

Yuxuan Zhang committed over 1 year ago

[Bugfix] Fix RuntimeError: Index put requires the source and destination dtypes match (#22065)

Chauncey committed over 1 year ago

[Kernel] Add support for block FP8 on SM120 (NVIDIA 5090 and RTX PRO 6000) (#22131)

Junhao Li committed over 1 year ago

Fix Flashinfer CUTLASS MOE Allgather (#21963)

Shu Wang committed over 1 year ago

Support Tensorrt-LLM MoE fp4 for low-latency (#21331)

Shu Wang committed over 1 year ago

Add ModelOpt Qwen3 nvfp4 support (#20101)

Zhiyu committed over 1 year ago

[PERF] Use pybase64 to more quickly decode prompt embeddings (#22469)

Andrew Sansom committed over 1 year ago

[ROCm] [V1] [SpecDec] Enable Speculative Decoding on ROCm V1 Engine (#21496)

TJian committed over 1 year ago

[Misc] normalize multiprocessing Queue usage (#22371)

Ning Xie committed over 1 year ago

Remove `from_dict` from `SpeculativeConfig` (#22451)

Harry Mellor committed over 1 year ago

[Frontend] Use engine argument to control MM cache size (#22441)

Cyrus Leung committed over 1 year ago