[Refactor] Simplify BOS/EOS token handling (#34435)

Cyrus Leung committed 9 months ago

[Bugfix] Remove assert that's no longer valid (#34443)

bnellnm committed 9 months ago

[BugFix] Add block_size validation for mamba cache align mode (#34445)

Harry Huang committed 9 months ago

Fix num_logprobs parameter description in sampler.py (#34451)

Zhuohan Li committed 9 months ago

[CI/Build] Update video URLs for testing (#34446)

Cyrus Leung committed 9 months ago

[Kernel] [Helion] [4/N] Add silu_mul_fp8 Helion kernel (#33373)

Yanan Cao committed 9 months ago

[Core] Profiler improvements and lazy initialization (#33198)

Jaewon committed 9 months ago

[Core] Add sleep level 0 mode with enqueue/wait pattern (#33195)

Jaewon committed 9 months ago

[Frontend] Enable generic structured_outputs for responses API (#33709)

Alec S committed 9 months ago

Use paged_attention_v1 for sliding window decode in rocm_aiter_fa (#34378)

Mengtao (Martin) Yuan committed 9 months ago

[Kernel] Support Flashinfer trtllm fused MoE non gated FP8 & NVFP4 (#33506)

amitz-nv committed 9 months ago

small adjustment to wvSplitKrc (#34410)

Hashem Hashemi committed 9 months ago

[Bugfix] Enforce DeepGEMM when using sparse_attn_indexer on CUDA (#34374)

Michael Goin committed 9 months ago

[ROCm][CI] Pin TorchCodec to v0.10.0 for ROCm compatibility (#34447)

Andreas Karatzas committed 9 months ago

[Voxtral Realtime] Refactor & Improve buffering logic (#34428)

Patrick von Platen committed 9 months ago

[Voxstral Realtime] Enable tests (#33803)

Patrick von Platen committed 9 months ago

[ROCm][quantization] improve OCP weight quant parser robust (#34431)

xuebwang-amd committed 9 months ago

[Bugfix] Remove broken raw url GGUF model loading support (#34433)

Isotr0py committed 9 months ago

Fix MoE for the Transformers modelling backend (#34436)

Harry Mellor committed 9 months ago

[Attention] Add FlashInfer Sparse MLA backend (#33451)

Matthew Bonanni committed 9 months ago