A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Quantization] Let ModelOpt MXFP8 layers load pre-processed weights Purpose (#57316)
siyu committed about 1 month ago
[Tests] Cover get_unhashed_block_ids_all_groups (#55928)
Aditya Ganesh Kumar committed about 1 month ago
[Docs] Explain pointer-alignment JIT specialization in Triton skill (#57669)
Woosuk Kwon committed about 1 month ago
[Bugfix][DSA] Avoid runtime JIT for offset candidate end buffers (#57667)
Woosuk Kwon committed about 1 month ago
[CI][AMD] Bump torchao to v18 for Python 3.14 (#53009)
Ryan Rock committed about 1 month ago
[Bugfix][CPU] Fix DeepSeek V4.1 import without Triton (#57654)
Jiang Wu committed about 1 month ago
[Bugfix][Multimodal] Frame the multi-modal hash digest input (#54283)
Hotragn Pettugani committed about 1 month ago
[Bugfix][Model] Fix Aria expert weight names and layout (#57487)
Jiang Wu committed about 1 month ago
[CI] Deflake pooling shards with GPU teardown fixtures between tests (#57371)
Kevin H. Luu committed about 1 month ago
[CI] Deflake MTEB score tests with per-model mteb_tol and reruns (#57364)
Kevin H. Luu committed about 1 month ago
[CI][Bugfix] Correct the Laguna DFlash acceptance-length reference (#57647)
Oxana Korzh committed about 1 month ago
[ROCm] Bump AITER to v0.1.22.post1 (#56885)
Micah Williamson committed about 1 month ago
[Core][Frontend] Bind KV-event publishers at port 0 and expose the bound endpoints (#55844)
hangangqiang committed about 1 month ago
[CI] Reclaim GPU memory between model initialization tests (#57362)
Kevin H. Luu committed about 1 month ago
[Bugfix][MRV2] Match fast-prefill padding to active LoRA batches (#56456)
waizuichougou committed about 1 month ago
[Feat][Model] Support encoder-side SWA-bounded replay for DeepSeek-V4.1-Flash (#56227)
Yifan Qiao committed about 1 month ago
[Bugfix][KVConnector] Make ExampleHiddenStatesConnector abort-safe (#56841)
ys2025-AI committed about 1 month ago
[Bugfix][MLA] Reserve sparse prefill buffers before KV cache sizing (#57575)
Lucas Wilkinson committed about 1 month ago
[ROCm][CI] Shard MI300 Entrypoints Integration (Pooling) (#57583)
Aarushi Jain committed about 1 month ago
[CI][ROCm] Deprecate DinD for MI250 test groups (#56162)
Andreas Karatzas committed about 1 month ago