A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 23:30 UTC
main
[VLM] Remove input processor from clip and siglip (#13165)
Isotr0py committed almost 2 years ago
[V1][core] Implement pipeline parallel on Ray (#12996)
Rui Qiao committed almost 2 years ago
[CI/Build] Allow ruff to auto-fix some issues (#13180)
Russell Bryant committed almost 2 years ago
Allow Unsloth Dynamic 4bit BnB quants to work (#12974)
Daniel Han committed almost 2 years ago
[Frontend] Move CLI code into vllm.cmd package (#12971)
Russell Bryant committed almost 2 years ago
LikeSundayLikeRain committed almost 2 years ago
[Build] Automatically use the wheel of the base commit with Python-only build (#13178)
Cody Yu committed almost 2 years ago
Simplify logic of locating CUDART so file path (#13203)
Lu Fang committed almost 2 years ago
[VLM] Implement merged multimodal processor for Mllama (#11427)
Isotr0py committed almost 2 years ago
[Bugfix][Example] Fix GCed profiling server for TPU (#12792)
Michael Goin committed almost 2 years ago
[NVIDIA] Support nvfp4 quantization (#12784)
Kaixi Hou committed almost 2 years ago
[perf-benchmark] cleanup unused Docker images and volumes in H100 benchmark instance (#12706)
Kevin H. Luu committed almost 2 years ago
[CI/Build] Ignore ruff warning up007 (#13182)
Russell Bryant committed almost 2 years ago
[V1][Bugfix] Copy encoder input ids to fix set iteration issue during VLM abort (#13173)
Murali Andoorveedu committed almost 2 years ago
[CI] Fix failing FP8 cpu offload test (#13170)
Michael Goin committed almost 2 years ago
[Bugfix] Allow fallback to AWQ from AWQMarlin at per-layer granularity (#13119)
Michael Goin committed almost 2 years ago
[CORE] [QUANT] Support for GPTQModel's `dynamic` quantization per module override/control (#7086)
Qubitium-ModelCloud committed almost 2 years ago
[CI/Build] Use mypy matcher for pre-commit CI job (#13162)
Russell Bryant committed almost 2 years ago
Introduce VLLM_CUDART_SO_PATH to allow users specify the .so path (#12998)
Lu Fang committed almost 2 years ago
[Misc] Delete unused LoRA modules (#13151)
Jee Jee Li committed almost 2 years ago