A high-throughput and memory-efficient inference and serving engine for LLMs
main
[ROCm][CI] Use a platform-independent GEMM in the merged-column fuser test (#56130)
djramic committed about 1 month ago
[Bugfix][V2] record_stream idx_mapping in the PP draft broadcast (#55745)
Chris Eastwood committed about 1 month ago
[CI][ROCm] Increase timeout for AMD MI355 Language Models (Standard) (#56114)
Aarushi Jain committed about 1 month ago
[ROCm][CI] Fix moe layer tests for fp8 dtype compatibility (#56106)
Divakar Verma committed about 1 month ago
[Bugfix][LoRA] Log when an adapter applies no weights (#55310)
Guancheng Fu committed about 1 month ago
[Bugfix] Make `mm_device_do_normalize` encoder-cudagraph safe (#55370)
cjackal committed about 1 month ago
[Spec Decode] Add NVFP4 DSpark gathered top-k projection (#55713)
Andrii Skliar committed about 1 month ago
[Pooling] Report LoRA adapter names in responses (#56004)
Taneem Ibrahim committed about 1 month ago
Fix ROCm AITER FP8 KV test tolerances. (#53590)
Aarushi Jain committed about 1 month ago
[Docs] Fix griffe docstring indentation warning in `SupportsMRoPE` (#56112)
Harry Mellor committed about 1 month ago
[Core][Model] Unify XD-RoPE into M-RoPE and derive the channel count (#56078)
Harry Mellor committed about 1 month ago
[Structured Output] Keep invalid structured-output requests from stopping the engine (#51450)
Clinton Thomas committed about 1 month ago
[Bugfix] Avoid FlexAttention recompiles when request counts change (#55888)
Andreas Karatzas committed about 1 month ago
[Rocm][Kimi-k3] Add pipeline_parallel support for the kimik3 model (#53664)
haic0 committed about 1 month ago
[ROCm] Bump AITER to v0.1.21.post2 (#55968)
Micah Williamson committed about 1 month ago
[Refactor][kv_offload]: rename `block`→`chunk` (#52615)
Ronen Schaffer committed about 1 month ago
[CI] Add e2e test for scale-out EC connector flow (#54973)
Nicolò Lucchesi committed about 1 month ago
[Perf] Improve BF16x3 router GEMM accuracy and make it default on sm100 (#55899)
Thien Tran committed about 1 month ago
[Qwen4Exp] Support UVA PLE-offload and Engram tensor parallelism (#54371)
Harry Huang committed about 1 month ago
[Bugfix][ROCm][DSv4] Skip launch_pdl=True JIT warmup when PDL is unsupported (#56035)
jimmy-adams committed about 1 month ago