Support S3 Sharded loading with RunAI Model Streamer (#16317)

omer-dayan committed over 1 year ago

[Performance][ROCm] Add skinny gemms for unquantized linear on ROCm (#15830)

Charlie Fu committed over 1 year ago

[V1] Remove additional_config check (#16710)

wangxiyuan committed over 1 year ago

[Kernel] Add expert_map support to Cutlass FP8 MOE (#16861)

Varun Sundar Rabindranath committed over 1 year ago

[Misc] Remove the chunked prefill warning for LoRA (#16925)

Jee Jee Li committed over 1 year ago

[ROCm] Add aiter tkw1 kernel for Llama4 fp8 (#16727)

kliuae committed over 1 year ago

[Bugfix]: fix issue with n>1 sampling on v1 requests overriding each other (#16863)

Jeffrey Li committed over 1 year ago

[BugFix][Spec Decode] No in-place update to draft probs (#16952)

Woosuk Kwon committed over 1 year ago

[Doc] Remove unnecessary V1 flag (#16924)

Cyrus Leung committed over 1 year ago

[TPU][V1] Enable Top-P (#16843)

Nicolò Lucchesi committed over 1 year ago

[V1] V1 FlashInfer Attention (#16684)

Michael Goin committed over 1 year ago

[TPU][V1] Capture multimodal encoder during model compilation (#15051)

Nicolò Lucchesi committed over 1 year ago

Update Qwen1.5-MoE-W4A16-compressed-tensors.yaml (#16946)

Michael Goin committed over 1 year ago

[TPU][V1] Implicitly adjust page size when there's SMEM OOM (#16871)

Chengji Yao committed over 1 year ago

[V1][Spec Decode] Handle draft tokens beyond max_model_len (#16087)

Woosuk Kwon committed over 1 year ago

[Core] Speed up decode by remove synchronizing operation in sampler (#16436)

Chanh Nguyen committed over 1 year ago

[Doc] mention how to install in CPU editable mode (#16923)

David Xia committed over 1 year ago

[doc] install required python3-dev apt package (#16888)

David Xia committed over 1 year ago

[XPU][Bugfix] minor fix for XPU (#15591)

Yan Ma committed over 1 year ago

Raise error for data-parallel with benchmark_throughput (#16737)

Kartik Ramesh committed over 1 year ago