A high-throughput and memory-efficient inference and serving engine for LLMs
main
[KV Connector][Offloading] Disable parallel-agnostic fs-tier cache on V2 model runner (#46044)
Itay Etelis committed 4 months ago
[Tests] Add Qwen3 streaming parser delta boundary cases (#45708)
Palaiologos1453 committed 4 months ago
[ROCm][Spec Decode] Fix probabilistic draft probs test attention backend (#45706)
stefankoncarevic committed 4 months ago
[CI]Enable mxfp4 lora test for ROCm platform (#43802)
qli88 committed 4 months ago
Humphrey committed 4 months ago
[ROCm][CI] move lora%N test to mi300 and gate (#45970)
Divakar Verma committed 4 months ago
[Frontend] Add Streaming Parser Engine and new GLM4.7/GLM5.1/GLM5.2 Parser (#45915)
Chauncey committed 4 months ago
[Bugfix][ROCm] Fix rocm_aiter_per_tensor_quant custom op aliasing (#45747)
Rohan Potdar committed 4 months ago
[Bugfix] Restore is_sym guard for zp in GPTQ/CT MoE to fix symmetric quant regression (#45656)
Yuwen Zhou committed 4 months ago
[Bugfix][Tool Parser] Handle non-finite numbers in coerce_to_schema_type (#43984)
Ashish Patel committed 4 months ago
Temporarily remove @markmc from CODEOWNERS (#46053)
Mark McLoughlin committed 4 months ago
[Model] Remove BambaForCausalLM (#45990)
Tiezhen WANG committed 4 months ago
Fix _riscv_supports_rvv_vlen128() to detect RVV on hardware without zvl flags (#43179)
lyd1992 committed 4 months ago
[Rust Frontend]: Add `/get_world_size` route with static parallel size (#44801)
Ashar committed 4 months ago
[Kernel] Add PDL support for DeepGEMM kernel (#46006)
Jee Jee Li committed 4 months ago
[ROCm][DSv4] Functional fixes for DeepSeek V4 on MI300X/MI325X (#45681)
Tuukka Sarvi committed 4 months ago
fix(quantization): Fix AWQ dequantize on Intel XPU and refactor AutoAWQ config (#42727)
AlexHuang committed 4 months ago
[CPU] Skip Triton kernel monkey-patches when Triton-CPU is available (#44991)
Jonathan Mamou committed 4 months ago
[Rust Frontend] Return model metadata fields in /v1/models (#45950)
Tahsin Tunan committed 4 months ago
Fix Stale Encoder Cache After Weight Update (#45093)
littlecircle0730 committed 4 months ago