A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 9, 14:45 UTC
main
[Doc] Add Qwen2.5 models to batch invariance tested models (#33016)
zhanqiuhu committed 10 months ago
[BugFix] Add env variable to control PDL in LoRA (#32836)
Jee Jee Li committed 10 months ago
[Bugfix] fix encoder cache hang in Qwen3VL (#32684)
JJJYmmm committed 10 months ago
[Docs] Fix Apple silicon include path in CPU installation docs (#32977)
7. Sun committed 10 months ago
[Perf][Kernel] Optimize FP4 quantization kernels (SM100F) (#32520)
Roberto L. Castro committed 10 months ago
[DOC] [ROCm] Update doc for v0.14.1 (#32998)
TJian committed 10 months ago
[Feature] add session based streaming input support to v1 (#28973)
Joshua Deng committed 10 months ago
Using max_loras + 1 to construct grid in fused_moe_lora (#32277)
yugong333 committed 10 months ago
[CPU] Improve CPU Docker build (#30953)
Maryam Tahhan committed 10 months ago
[CPU Backend][BugFix] Fix failing Darwin pipelines (#33002)
Fadi Arafeh committed 10 months ago
[Tests] Replace flaky sleep with polling in test_background_cancel (#32986)
7. Sun committed 10 months ago
[MLA] Fuse cat and qaunt for fp8 kv-cache (#32950)
Lucas Wilkinson committed 10 months ago
Update CPU doc according to feedback (#32963)
Louie Tsai committed 10 months ago
[Bugfix]: resolve torch.compile cache conflict between mm_encoder_tp_modes (#32842)
Hiroken. committed 10 months ago
[EncoderCacheManager] Remove unnecessary copy (#32800)
Lukas Geiger committed 10 months ago
feat: Complete LoRA support for MiniMaxM2 Fixes #32736 (#32763)
david guan committed 10 months ago
[Models]: Make Multimodal config implicit in ViT implementation (#31972)
Isotr0py committed 10 months ago
[Bugfix] Fix E2E latency calculation and add warmup support in mm_processor benchmark (#32646)
Hiroken. committed 10 months ago
[Perf] Cache exc.errors() result in validation exception handler (#32984)
7. Sun committed 10 months ago
[UX] Deduplicate sampling parameter startup logs (#32953)
Cyrus Leung committed 10 months ago