A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 06:45 UTC
main
[FIX] Fix docker build error (#1831) (#1832)
Allen committed almost 3 years ago
Avoid multiple instantiations of the RoPE class (#1828)
Jee Li committed almost 3 years ago
Refactor Worker & InputMetadata (#1843)
Woosuk Kwon committed almost 3 years ago
Disable Logs Requests should Disable Logging of requests. (#1779)
Michael McCulloch committed almost 3 years ago
[Docs] Add information about using shared memory in docker (#1845)
Simon Mo committed almost 3 years ago
Woosuk Kwon committed almost 3 years ago
Better integration with Ray Serve (#1821)
FlorianJoncour committed almost 3 years ago
[Fix] Fix RoPE in ChatGLM-32K (#1841)
Woosuk Kwon committed almost 3 years ago
[FIX] Fix formatting error in main branch (#1822)
Zhuohan Li committed almost 3 years ago
Zhuohan Li committed almost 3 years ago
Use the type BlockTable (#1791)
explainerauthors committed almost 3 years ago
[FIX] Fix class naming (#1803)
Zhuohan Li committed almost 3 years ago
Woosuk Kwon committed almost 3 years ago
Correct comments in parallel_state.py (#1818)
explainerauthors committed almost 3 years ago
Init model on GPU to reduce CPU memory footprint (#1796)
ljss committed almost 3 years ago
Added echo function to OpenAI API server. (#1504)
Yunmo Chen committed almost 3 years ago
Woosuk Kwon committed almost 3 years ago
[Build] Avoid building too many extensions (#1624)
Yanming W committed almost 3 years ago
Fix repetition penalty aligned with huggingface (#1577)
ljss committed almost 3 years ago
Set top_p=0 and top_k=-1 in greedy sampling (#1748)
ljss committed almost 3 years ago