A high-throughput and memory-efficient inference and serving engine for LLMs
main
[ROCm][CI] Run Basic Correctness Sleep Mode on MI300 for now (#59262)
Aarushi Jain committed 22 days ago
[KVConnector][MoRIIO] Support K3 DSpark hybrid READ (#57700)
YukioZzz committed 22 days ago
[CI] Mint the CRCR report's OIDC token after the wait, not before (#59259)
Andrey Talman committed 22 days ago
[Frontend] Attach resolved logprobs to streaming derender chunks (#55029)
Shimi Bandiel committed 22 days ago
[CI][ROCm] Increase timeouts for AMD MI300 jobs near their limits (#59227)
djramic committed 22 days ago
[Bugfix][DP] add_dp_placement_groups does not require ray[default] (#57648)
Chanbin Lim committed 22 days ago
[Mypy] Fix mypy typing for Zamba2 models (#58255)
Ashraf Bhuiyan committed 22 days ago
Chaitanya Sri Krishna Lolla committed 22 days ago
[Bugfix][Rust Frontend] Skip engine-derived metrics under `--disable-log-stats` (#59205)
Bugen Zhao committed 22 days ago
[Fast Start] Charge daemon-held weights against `gpu_memory_utilization` (#57298)
siyu committed 22 days ago
[Bugfix] Fix standalone torch.compile cache loading after relocation (#52142)
Jung Jiyu committed 22 days ago
[Bugfix][Frontend] Return 400 for malformed RL dev route bodies (#59236)
aoshen02 committed 22 days ago
[Bugfix] Fix TorchCodec audio IO correctness (#58364)
Isotr0py committed 22 days ago
[CI][ROCm] Fix stale basic_correctness path in Model Runner V2 Distributed (#59201)
stefankoncarevic committed 22 days ago
[Mypy] Fix mypy typing for Whisper models (#58254)
Ashraf Bhuiyan committed 22 days ago
[Bugfix][Quantization] Stop sleep(level=2) from zeroing compressed-tensors KV scales (#57163)
AlanFokCo committed 22 days ago
[Bugfix][MiMo] Declare embedding_fields so an EPD pair can serve images (#58938)
Giulio De Pasquale committed 22 days ago
[Bugfix] Profile maximum DeepSeek V4.1 vision features (#57898)
ahmed xijiaat committed 22 days ago
[Bugfix][CI] Assert the logger call the Anthropic merge warning makes (#59202)
stefankoncarevic committed 22 days ago
[MM] Fix compiled ViT attention output layouts (#58182)
Jakub Zakrzewski committed 22 days ago