A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 05:30 UTC
Require triton >= 2.1.0 (#2746)
Co-authored-by: yangrui1 <yangrui@lanjingren.com>
whyiug committed
Feb 5, 2024 at 15:07 UTC
c9b45adeeb0e5b2f597d1687e0b8f24167602395
1 file changed
+1
requirements.txt
+1
@@ -11,3 +11,4 @@ uvicorn[standard]
11
pydantic >= 2.0 # Required for OpenAI server.
12
aioprometheus[starlette]
13
pynvml == 11.5.0
14
+triton >= 2.1.0