A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 18:15 UTC
main
[multi-step] Raise error if not using async engine (#7703)
William Lin committed about 2 years ago
[Bugfix] Pass PYTHONPATH from setup.py to CMake (#7730)
sasha0552 committed about 2 years ago
[Bugfix][Frontend] Fix Issues Under High Load With `zeromq` Frontend (#7394)
Robert Shaw committed about 2 years ago
[Bugfix] chat method add_generation_prompt param (#7734)
Brian Li committed about 2 years ago
[Spec Decoding] Use target model max length as default for draft model (#7706)
Nick Hill committed about 2 years ago
[Bugfix][Hardware][CPU] Fix `mm_limits` initialization for CPU backend (#7735)
Isotr0py committed about 2 years ago
LI MOU committed about 2 years ago
[BugFix] Avoid premature async generator exit and raise all exception variations (#7698)
Nick Hill committed about 2 years ago
[Bugfix] Mirror jinja2 in pyproject.toml (#7723)
sasha0552 committed about 2 years ago
[mypy] Enable following imports for entrypoints (#7248)
Cyrus Leung committed about 2 years ago
[Doc] Section for Multimodal Language Models (#7719)
Roger Wang committed about 2 years ago
[Model] Add AWQ quantization support for InternVL2 model (#7187)
Isotr0py committed about 2 years ago
[ci] try to log process using the port to debug the port usage (#7711)
youkaichao committed about 2 years ago
[Core] Pipe `worker_class_fn` argument in Executor (#7707)
Antoni Baum committed about 2 years ago
[ci][test] adjust max wait time for cpu offloading test (#7709)
youkaichao committed about 2 years ago
[Intel GPU] fix xpu not support punica kernel (which use torch.library.custom_op) (#7685)
Kunshang Ji committed about 2 years ago
[Core] Add `AttentionState` abstraction (#7663)
Antoni Baum committed about 2 years ago
[Misc] Add jinja2 as an explicit build requirement (#7695)
Lucas Wilkinson committed about 2 years ago
[CI/Build] Pin OpenTelemetry versions and make errors clearer (#7266)
Ronen Schaffer committed about 2 years ago
[Hardware] [Intel GPU] refactor xpu worker/executor (#7686)
Kunshang Ji committed about 2 years ago