A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 02:00 UTC
main
[misc] fix cross-node TP (#12166)
youkaichao committed almost 2 years ago
[torch.compile] disable logging when cache is disabled (#12043)
youkaichao committed almost 2 years ago
[AMD][FP8] Using MI300 FP8 format on ROCm for block_quant (#12134)
Gregory Shtrasberg committed almost 2 years ago
[Bugfix] Mistral tokenizer encode accept list of str (#12149)
Kunshang Ji committed almost 2 years ago
[Bugfix] Fix score api for missing max_model_len validation (#12119)
Wallas Henrique committed almost 2 years ago
[core] allow callable in collective_rpc (#12151)
youkaichao committed almost 2 years ago
[CI/Build][CPU][Bugfix] Fix CPU CI (#12150)
Li, Jiang committed almost 2 years ago
[Misc][LoRA] Improve the readability of LoRA error messages (#12102)
Jee Jee Li committed almost 2 years ago
[V1] Move more control of kv cache initialization from model_executor to EngineCore (#11960)
Chen Zhang committed almost 2 years ago
[ROCm][MoE] moe tuning support for rocm (#12049)
Divakar Verma committed almost 2 years ago
[Misc] Add deepseek_vl2 chat template (#12143)
Isotr0py committed almost 2 years ago
[BugFix] add more `is not None` check in VllmConfig.__post_init__ (#12138)
Chen Zhang committed almost 2 years ago
[Bugfix] Fix issues in CPU build Dockerfile (#12135)
Yuan Tang committed almost 2 years ago
[Doc] Add instructions on using Podman when SELinux is active (#12136)
Yuan Tang committed almost 2 years ago
[CI]add genai-perf benchmark in nightly benchmark (#10704)
Kunshang Ji committed almost 2 years ago
[Bugfix] Fix a path bug in disaggregated prefill example script. (#12121)
Kuntai Du committed almost 2 years ago
[Bugfix] Set enforce_eager automatically for mllama (#12127)
Chen Zhang committed almost 2 years ago
[Model] Add support for deepseek-vl2-tiny model (#12068)
Isotr0py committed almost 2 years ago
[misc] Add LoRA kernel micro benchmarks (#11579)
Varun Sundar Rabindranath committed almost 2 years ago
[Bugfix] Fix max image feature size for Llava-one-vision (#12104)
Roger Wang committed almost 2 years ago