Remove deprecated `--rope-scaling` and `--rope-theta` (#28006)

Harry Mellor committed about 1 year ago

[AsyncScheduling] Don't schedule past request max_tokens (#27922)

Nick Hill committed about 1 year ago

[PerfFix] Avoid separate thread for MP executor shm spin (#28012)

Nick Hill committed about 1 year ago

[model] Add support for openPangu_Ultra_MoE (#27521)

yt0428 committed about 1 year ago

[PERF] Decouple projections from GDN custom op (#27512)

Vadim Gimpelson committed about 1 year ago

[Perf] SM100 - add swap AB optimization to CUTLASS FP8 GEMM (#27284)

lyrisz committed about 1 year ago

[BugFix] Fix incorrect preallocated sampled_token_ids tensor size (#28025)

Nick Hill committed about 1 year ago

[Kernels] Isolate modular kernel code from FusedMoEMethodBase subclasses. (#27123)

bnellnm committed about 1 year ago

[Model] add optimal triton fused moe configs for NemotronH MoE (#27967)

tomeras91 committed about 1 year ago

[Model][Bugfix] fix pipeline parallelism support for NemotronH (#27968)

tomeras91 committed about 1 year ago

[Core] Enable StatLogger in LLMEngine (#28020)

Zhuohan Li committed about 1 year ago

Support using Int4PreshuffledTensor after loading (#26066)

Jerry Zhang committed about 1 year ago

Load tuned fused_moe_lora shrink and expand kernel configs separately (#27435)

yugong333 committed about 1 year ago

[BugFix][Performance] Restore flashinfer autotuning for all scenarios (#27904)

Varun Sundar Rabindranath committed about 1 year ago

[CI/Build] Fix OpenAI API correctness on AMD CI (#28022)

Zhewen Li committed about 1 year ago

[Model] fix ernie45 reasoning_parser (#27973)

CSWYF3634076 committed about 1 year ago

[KV Connector] Make KVCacheConfig an explicit constructor argument (#27887)

Mark McLoughlin committed about 1 year ago

[CI/Build] Update LM Eval Version in AMD CI (#27944)

Zhewen Li committed about 1 year ago

[CPU]Improve dynamic 4bit moe performance (#27240)

xiangze-arm committed about 1 year ago

[Bug] Batch invariant: Fix flash attn MLA `RuntimeError: scheduler_metadata must have shape (metadata_size)` (#27884)

Wentao Ye committed about 1 year ago