[Perf] Reduce MLA CPU overheads in V1 (#14384)

Lucas Wilkinson committed over 1 year ago

[Bugfix] Correctly call `cudaProfilerStop` in benchmarks script (#14183)

Brayden Zhong committed over 1 year ago

[Doc] Fix a typo (#14385)

Daniel Li committed over 1 year ago

[Hardware][TPU]Enable ragged paged attention kernel and resolve recompilation issue (#14310)

Chengji Yao committed over 1 year ago

[Docs] Add nsight guide to profiling docs (#14298)

Michael Goin committed over 1 year ago

[V1][Bugfix] Standardize quantized kv cache rejection for attention backends (#14221)

Michael Goin committed over 1 year ago

[Bug] Fix Attention when ignored in by quant_method (#14313)

Michael Goin committed over 1 year ago

[Bugfix] Fix use_direct_call condition in FusedMoE layer for (#14382)

Tyler Michael Smith committed over 1 year ago

[Kernel] Add needs_fixed_stride_order tag to most GEMMs (#14306)

Tyler Michael Smith committed over 1 year ago

[CI] Disable spawn when running V1 Test (#14345)

Thomas Parnell committed over 1 year ago

[CI/Build] Use uv python for docker rather than ppa:deadsnakes/ppa (#13569)

Michael Goin committed over 1 year ago

[Distributed] Add enable_expert_parallel arg (#14305)

Tyler Michael Smith committed over 1 year ago

[V1] Do not detokenize if sampling param detokenize is False (#14224)

Himanshu Jaju committed over 1 year ago

Fix mla prefill context performance (#13897)

Ying Zhong committed over 1 year ago

Add authors to license header. (#14371)

Thomas Parnell committed over 1 year ago

Adding cpu inference with VXE ISA for s390x architecture (#12613)

Dilip Gowda Bhagavan committed over 1 year ago

Reinstate `best_of` for V0 (#14356)

Harry Mellor committed over 1 year ago

[RLHF] use worker_extension_cls for compatibility with V0 and V1 (#14185)

youkaichao committed over 1 year ago

[Doc] Fix date typo in README.md (#14366)

Jitse Klomp committed over 1 year ago

[Core] Don't use cache during multi-modal profiling (#14336)

Cyrus Leung committed over 1 year ago