[Multimodal] Update Tensor schema test to cover arbitrary shape mm inputs (#22867)

Isotr0py committed about 1 year ago

[Misc] Add --save-dir option to benchmark_moe (#23020)

Jee Jee Li committed about 1 year ago

[BugFix] Add support for loading prompt embeds tensors serialized on unavailable devices and sparse tensors (#22962)

Andrew Sansom committed about 1 year ago

[Build] Env var to disable sccache (#22968)

Lucas Wilkinson committed about 1 year ago

[XPU]avoid circular import during XPU init (#23017)

Kunshang Ji committed about 1 year ago

[misc] nsys profile output kernel classifier and visualizer (#22971)

Grace Ho committed about 1 year ago

[V1] support min_tokens for detokener (#22014)

Calvin Chen committed about 1 year ago

[Frontend] Avoid list copies in `serving_chat.py` (#22947)

Nick Hill committed about 1 year ago

[Bugfix] Fix DeepSeek MTP (#22934)

Benjamin Chislett committed about 1 year ago

[Kernel] Simplify `get_kv_cache_layout` and cache `use_trtllm_attention` env-dependent bit (#22735)

Nicolò Lucchesi committed about 1 year ago

[Misc] Support passing multiple request ids at once to `AsyncLLM.abort()` (#22944)

Nick Hill committed about 1 year ago

[BugFix] Make `run_once` thread-safe (#22978)

Yichen Yan committed about 1 year ago

Support multiple attention groups for KV sharing (#22672)

Yong Hoon Shin committed about 1 year ago

[v1] Move block_hashes from KVCacheManager to Request.block_hashes (#19728)

Or Ozeri committed about 1 year ago

[BugFix] Handle case where async utility call is cancelled (#22996)

Nick Hill committed about 1 year ago

[Structured Outputs] [Bug] Fix misalignment in apply_grammar_bitmask causing unintended masking and NaN logits (#22963)

rishitdholakia13 committed about 1 year ago

ci: Add CUDA + arm64 release builds (#21201)

Eli Uriegas committed about 1 year ago

[BugFix] Fix regression caused by mamba state dtype PR (#22998)

Thomas Parnell committed about 1 year ago

[Attention] FA3 Attention Sinks Perf Boost (#22478)

Lucas Wilkinson committed about 1 year ago

minor: zero workspace buffer init for flashinfer trtllm-gen attn (#22603)

eigen committed about 1 year ago