A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Misc] Mistral label alert (#47132)
Nicolò Lucchesi committed 4 months ago
[CI/Build] Add CPU test dependency pre-commit hooks (#47032)
Li, Jiang committed 4 months ago
[Bugfix][Responses] Set completed status for Harmony function calls (#46945)
Aman Paswan committed 4 months ago
[Frontend] Add Streaming Parser Engine and new Kimi k2.5/k2.6/k2.7 Parser (#46610)
Chauncey committed 4 months ago
[XPU] Optimize XPU worker shutdown logic to prevent resource leak (#46433)
Chaojun Zhang committed 4 months ago
[ROCm][Ray][CI] Keep assigned GPU visible for weight transfer (#47000)
Andreas Karatzas committed 4 months ago
[Bugfix][ROCm] Preserve MoE weight padding for unquantized Triton path (#46381)
Uros Markovic committed 4 months ago
[Platform] Replace `torch.cuda.mem_get_info` with `torch.accelerator.get_memory_info` (#44825)
Kunshang Ji committed 4 months ago
Dakai An committed 4 months ago
[Bugfix] Raise VLLMValidationError for non-integer logit_bias keys (#46612)
Muhammad Fawaz committed 4 months ago
[ROCm][CI][Multimodal] Use ROCm-aware FA availability check for Unlimited-OCR (#47004)
Andreas Karatzas committed 4 months ago
[Rust Frontend] Start current wave for a stale DP FirstRequest (#46833)
Blas Rodriguez Irizar committed 4 months ago
[Bugfix] Use larger workspace size for Flashinfer MLA LSE (#47074)
Wei Zhao committed 4 months ago
[Bugfix][Rust Frontend] Reject prompt_logprobs for streaming generate (#46839)
Reid committed 4 months ago
[Rust Frontend] Avoid LoRA registry scans without active LoRA requests (#47040)
Reid committed 4 months ago
Secondary tier implementation for PD disaggregation (#42285)
liranschour committed 4 months ago
[LoRA] Add language-backbone LoRA support for MiniCPM-V 4.6 (#46740)
linitra24 committed 4 months ago
[XPU][UT]Fix xpu pass_config.fuse_norm_quant assert issue (#46804)
Yejing Lai committed 4 months ago
[Bugfix] Propagate default stop_token_ids to per-request SamplingParams (#35076)
ganesh committed 4 months ago
fix(config): reject negative max_logprobs (except -1) and long_prefill_token_threshold (#44070)
hcl committed 4 months ago