A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Rust Frontend] Bump llm-multimodal version (#47530)
Isotr0py committed 4 months ago
Move Roberta remaining nn.Embedding to VocabParallelEmbedding (#47452)
Max de Bayser committed 4 months ago
[Doc] Fix VLM2Vec benchmark chat template path (#47517)
Kalyanam Dewri committed 4 months ago
Add Triton Backend for Unlimited-OCR R-SWA (#47102)
Dakai An committed 4 months ago
[CI] Enable sccache for Rust build under CUDA/ROCm (#45246)
Bugen Zhao committed 4 months ago
[Rust Frontend] add repetition_detection support to sampling params (#46684)
yangyang-cs95 committed 4 months ago
New stable abi cleanup (#46656)
Chris Leonard committed 4 months ago
[XPU][CI]Fix dependency typo in Intel GPU CI (#47510)
xiangdong committed 4 months ago
[CPU][Build] Enable oneDNN ITT task collection by default for CPU primitive-level profiling (#47467)
Evgeny Parshutin committed 4 months ago
[XPU][CI]Mv huggingface cache to larger disk in Intel GPU CI (#47405)
xiangdong committed 4 months ago
[Rust Frontend] Recover buffered text from incomplete tool calls at EOS (#47289)
Reid committed 4 months ago
[Bugfix][Rust Frontend] Tolerate out-of-vocab prompt ids in detokenizer (#44682)
Ting SUN committed 4 months ago
[Bugfix][Gemma4] Keep image bidirectional attention within the sliding window (#47217)
Luciano Martins committed 4 months ago
[BugFix] Derive FlashInfer Q dtype from resolved per-group builder state (#47485)
Michael Goin committed 4 months ago
[CI] Pin modelscope version to fix test breakage (#47465)
Nick Hill committed 4 months ago
[ModelRunner V2][BugFix] Free all model refs on shutdown (#47483)
Nick Hill committed 4 months ago
[CI] intel CI: add quantization and awq case for xpu (#46456)
wenjun liu committed 4 months ago
Add Laguna XS.2.1 DFlash drafter support (#46853)
adamkbaranowski committed 4 months ago
[ROCm][CI] Adding extract hs 2gpu (#47482)
Andreas Karatzas committed 4 months ago
[ROCm][CI] Adding qwen3 dp4 eplb (#47480)
Andreas Karatzas committed 4 months ago