A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 03:00 UTC
[Bugfix] Add `__init__.py` files for `vllm/core/block/` and `vllm/spec_decode/` (#3798)
Michael Goin committed
Apr 2, 2024 at 12:35 UTC
b321d4881b273b2b7085093816443a718c62d24a
2 files changed