A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Bugfix][LoRA] Add multimodal module mapping for Muse-Glimmer (#53513)
linitra24 committed 2 months ago
[LoRA] Add Qwen3-Omni multimodal LoRA support (#52786)
linitra24 committed 2 months ago
[Perf] Tune FlashInfer all-reduce selection on SM103 (#53318)
Summer Yang committed 2 months ago
[Bugfix][XPU] Skip oneCCL warm-up all_reduce when world_size == 1 (#52389)
DaeHyun Sung committed 2 months ago
[LoRA] feat: Support LoRA for DeepSeek V4 (#53361)
ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 committed 2 months ago
[Model] Add FP8 quantization support for ModernBERT (#53101)
ThunGuo committed 2 months ago
Reject oversized media before fully downloading it (#51896)
Clinton Thomas committed 2 months ago
feat: add SSE keep-alive comments for idle streaming responses (#51034)
Dieter Schwarzmann committed 2 months ago
Add MTP support for Nemotron VL models (#53121)
danisereb committed 2 months ago
[Misc] Use VLLMValidationError in Cohere request validation (#52467)
Frank committed 2 months ago
[Model] Fix KV cache layout and optimize Dots3 NOTE Omni encoders (#53460)
范裕达 committed 2 months ago
[RL] P2P RDT weight sync (#43375)
Aaron Hao committed 2 months ago
[Misc] Use VLLMValidationError in offline inference input validation (#53432)
Jensen Chen committed 2 months ago
Add routed expert loading for gpt-oss (#52209)
Wyett committed 2 months ago
[CI/Build] Pin Cython below 3.3 for arm64 tilelang sdist (#53358)
Kevin H. Luu committed 2 months ago
[MM] Simplify prompt updates: replace `PromptSeq` with `list[int]` (#53372)
Cyrus Leung committed 2 months ago
[Bugfix][Model] Mistral3: fix image placeholder grid for processor size overrides (#52874)
anmolgupt committed 2 months ago
[CI][AMD] Honor single-node Docker workload timeout (#52547)
Andreas Karatzas committed 2 months ago
[Rust Frontend][RL]: report engine world size over gRPC (#53204)
Biswa Panda committed 2 months ago
[ROCm][CI] Restore attention coverage after KV-cache layout refactor (#53351)
Andreas Karatzas committed 2 months ago